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Preface 



This book is intended as a text on mathematical modehng for undergraduate 
and graduate students of mathematics, engineering, economics, finance, biology, 
chemistry, and physics. The material follows the author's undergraduate teaching 
of Introduction to Mathematical Modeling and graduate teaching of Deterministic 
and Stochastic Mathematical Modeling over the last ten years. The first character- 
istic feature of this text is the systematic development of deterministic and sto- 
chastic modeling approaches. Such a development is relevant because most real- 
world processes involve randomness. The consideration of stochastic methods 
enables a comprehensive understanding, for example, of the basis of optimal 
deterministic models and how closed deterministic equations can be obtained. The 
second characteristic feature of this text is the systematic discussion of single 
problems: the analysis of observations, characteristic properties and changes of 
one variable, and the laws that govern the evolution of one and several variables. 
An alternative approach would be the simultaneous discussion of difference and 
differential equations, or the simultaneous discussion of differential equations for 
one and several variables. The latter approach would make the presentation more 
difficult to understand because of the simultaneous explanation of the laws of 
stochastic evolution and meaning of stochastic concepts, or the explanation of the 
laws of stochastic evolution in notation for several variables. The third character- 
istic feature of this text is a hierarchical development of models (if possible). 
Examples for this approach are the discussion of statistically most-likely probabil- 
ity density functions, the relations between difference and differential equations, 
the Brownian motion model and diffusion model, the delay logistic model, non- 
Markovian and Markovian velocity models, the nonlinear and linear pendulum 
motion, and the derivation of equations for fluid dynamics in Chaps. 4—10, 
respectively. Such a systematic discussion of models is relevant to see the usual 
hierarchical structure of models and the range of applicability of certain models. 
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A fourth characteristic feature of this text is the attempt to provide a complete 
presentation. Most of the derivations are presented by providing all the required 
details (exercises are used to provide additional details). Discussions of problems 
are provided as complete as possible (see the discussion of Lorenz's equations). 

The book enables the illustration of the application of the developed concepts 
by 570 exercise questions organized in 220 problems. The detailed solutions to all 
questions are given in the Instructor's Solutions Manual, which can be provided to 
instructors by the publisher. The exercises are given at the end of each chapter. 
For example, the notation "Exercise 4.2.4" refers to the fourth problem related to 
Sect. 4.2. The exercises provided here are much more than sufficient for the 
assignment of homework. In many cases, it is possible to create additional home- 
work by minor modifications of the suggested problems. 

Apart from basic algebra, a first essential prerequisite for following the mate- 
rial presented in this book is single-variable calculus (I and II): students have to be 
familiar with differentiation and integration, the calculation of local extreme 
values, and the Taylor series. Complex numbers will be used in Chaps. 5, 7, and 9. 
Prior knowledge of differential equations is helpful for understanding Chaps. 7- 
10, but it is not a requirement. The solution of separable differential equations, 
which is the usual type of equation involved, is explained in detail in Chap. 7. 
Chapter 9 involves the use of two-by-two and three-by-three matrices. Chapter 10 
applies multivariate calculus concepts, e.g., integrals over functions of several 
variables, partial derivatives, and partial differential equations. However, these 
developments will be shown to represent simple extensions of single-variable 
calculus and related methods (for the solution of the Fokker-Planck equation). 



Table 1 Overview of chapters and questions addressed in this book. 



1. Determ. Analysis Observations 

How can we develop models that 
describe the trend of observations? 

3. Determ. States 

What are characteristic properties 
of a deterministic variable? 



2. Stoch. Analysis Observations 

How can we find optimal models 
that account for data randomness? 

4. Stoch. States 

Wliat are characteristic properties 
of a random variable? 



5. Determ. Changes 

What are characteristic changes 
of a deterministic variable? 

1. Determ. Evolution 

What are the laws of the evolution 
of one deterministic variable? 

9. Determ. Multivariate Evolution 

What are the laws of the evolution 
of several deterministic variables? 



6. Stoch. Changes 

What are characteristic changes 
of a random variable? 

8. Stoch. Evolution 

What are the laws of the evolution 
of one random variable? 

10. Stoch. Multivariate Evolution 

What are the laws of the evolution 
of several random variables? 



Preface IX 

A second essential prerequisite for applying the concepts developed in this text 
is basic knowledge of a software package (like Matlab, Mathematica, Maple) that 
can be used to perform relatively simple numerical calculations. Students should 
be able to read in given data, apply simple data transformations, analyze the data, 
and show the results of computations in figures. Examples for calculations that 
students should be able to perform after the explanation of the corresponding con- 
cepts are the plot of model functions in comparison to random data, the calcula- 
tion of a probability density function, and the numerical solution of ordinary dif- 
ferential equations. From the author's view point, such numerical exercises do 
essentially contribute to the understanding of students. In fact, only the exercises 
of Chaps. 1, 2, 4, and 6 (and two exercises of Chap. 9) require the use software. 

A first possibility of using this text is to apply the first four chapters (the upper 
box in Table 1) for the teaching of a three-credit undergraduate course Introduc- 
tion to Mathematical Modeling. This course would be focused on a basic under- 
standing of how simple analytical functions can be used for the modeling of many 
real-world problems. The students learn to describe the trend of observations, to 
deal with the need to consider several variables, to design optimal models, and to 
assess randomness. The level of this course would be comparable to Calculus II. 
Instead of covering all the sections of Chap. 4, it is a good alternative to focus on 
models for probability density functions (Sect. 4.3) combined with Sect. 10.2, 
which explains (in difference to the approach used in Chap. 2) the development of 
optimal models from a probability perspective. Another possibility is given by 
covering Chap. 5 instead of Chap. 4. 

A second possibility of using this text is to apply Chaps. 5, 7, and 9 (the lower 
left-hand side box in Table 1) for the teaching of a three-credit undergraduate and 
graduate course Deterministic Mathematical Modeling. Such a course would be 
focused on a basic understanding of how real-world processes can be modeled on 
the basis of deterministic ordinary differential equations. The students learn about 
the application, typical advantages and disadvantages of difference and differen- 
tial equations, the characteristic changes and evolution laws of deterministic pro- 
cesses, and the ways to model and analyze the interaction of processes. The level 
of this course would be comparable to Applied Differential Equations I and II 
courses. The difference to usual Applied Differential Equations courses would be 
the clear focus on the application of typical differential equations. The discussion 
of Chaps. 5, 7, and 9 combined with additional practice problems taken from the 
exercises (or taken from Haberman 1977, Fulford et al. 1997, Edelstein-Keshet 
2005, Brannan & Boyce 2007, Nagle et al. 2008, Boyce & DiPrima 2009) does 
provide sufficient material for a one-semester course. A possibility to provide a 
broader perspective would be the additional discussion of Sects. 3.2 and 3.3 
related to the application of dimensional analysis in the beginning of this course. 
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A third possibility of using this text is to apply Chaps. 6, 8, and 10 (the lower 
right-hand side box in Table 1) for the teaching of a three-credit graduate course 
Stochastic Mathematical Modeling. Required knowledge about basic properties of 
random variables can be provided by involving a part of Chap. 4 (Sects. 4.2 - 4.4). 
Such a course would be focused on a basic understanding of how random real- 
world processes can be analyzed and modeled on the basis of stochastic ordinary 
differential equations. The students learn about the relationship between stochastic 
differential equations and evolution equations for probability density functions, 
the characteristic features of Monte Carlo simulation, the way to develop 
stochastic models, and the use of stochastic methods for developing consistent 
models for multi-scale processes. Because of the inclusion of stochastic methods, 
ordinary and partial differential equations, the level of this course would be higher 
than the levels of the two courses described above. A way to exclude multivariate 
calculus concepts is to focus the presentation on Chaps. 4, 6, 8, and Sects. 10.2 
and 10.3. There are many ways to illustrate the use of stochastic methods in 
particular applications, for example with regard to financial mathematics 
(Buchanan 2008), biology (Allen 2003), turbulent reacting flows (Haworth 2010), 
flow in porous media (Tyagi et al. 2008), two-phase flows (Minier & Peirano 
2001), or many other applications (Kloeden & Platen 1992). 

It is a pleasure to thank many people for significant support over many years: 
In particular, I am profoundly grateful to the Professors F. Jafari, P. Jenny, P. Givi, 
J. Naughton, D. Roekaerts, B. Shader, S. Sritharan, and the Drs. H. Gopalan and M. 
StoUinger. My sincere appreciation and thanks are expressed to all my colleagues 
at the Mathematics Department (University of Wyoming) for the pleasant atmos- 
phere and a lot of help. I am very thankful to Professor G. Katul (Duke University, 
Durham, NC) for providing the instantaneous velocity and temperature data 
measured in the atmospheric surface layer (Chu et al. 1996), which were used for 
the illustration of real probability density functions in Sect. 4.5. Special thanks go 
to Dr. G. Turner (CSIRO, Sustainable Ecosystems, Australia) for providing the 
predictions of the World3 model, which describes the evolution of the global eco- 
nomic system (Turner 2008). These model predictions were used in Sect. 7.5 for 
the discussion of oscillations and collapse in population ecology. Many thanks 
also go to Dr. Ch. Baumann (Springer, Heidelberg) for his understanding and the 
good collaboration regarding the production of this text. For copyediting and care- 
ful text corrections I am grateful to Theodor C.H. Cole. And most of all, I want to 
deeply thank my wife Petra. 

Stefan Heinz 

Laramie, Wyoming 

May 20 11 
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1 Deterministic Analysis of Observations 



A usual modeling problem is given by the case that there are some relevant 
observations which may be obtained by measurements, and we would like to have 
a relatively simple mathematical function that provides a model for the obser- 
vations. The purpose of finding such a function is to obtain a qualitative and 
quantitative understanding of observations, which is helpful for reacting to the 
observations in an appropriate way. The process of developing a mathematical 
model involves several basic steps. The first step is to present the problem as 
simple as possible, for example by transformations of data. The second step is to 
use modeling concepts to derive various reasonable models. The third step is to 
evaluate the models obtained in order to identify the optimal model. A model that 
is developed on the basis of observations represents the same information as given 
by the observations. However, a model should also provide an additional benefit. 
Thus, the fourth step is to demonstrate the advantage of the model development by 
deriving valuable conclusions that are not directly given by the observations. 

The basic four steps of the modeling process will be illustrated in this chapter. 
We do only consider deterministic models here, and we only address the problem 
of developing models for observations that depend on one variable (models for 
several variables that can account for randomness will be considered in other 
chapters). The sort of problems considered in this chapter will be explained in 
Sect. 1.1. Sections 1.2-1.5 describe ways for dealing with the four basic steps of 
the mathematical modeling process. Section 1.2 explains data transformations for 
obtaining linear relations. Section 1.3 presents polynomial models that provide a 
basis for the development of simple analytical models. Section 1 .4 addresses the 
evaluation of models by comparing four models for the development of the U.S. 
population. Section 1.5 addresses the modeling of global warming to show how 
models can be used for deriving conclusions regarding the driving mechanism for 
observations. The discussions in this chapter will be summarized in Sect. 1.6. 



S. Heinz, Mathematical Modeling, D0\ 10.1007/978-3-642-203 11-41, 
© Springer- Verlag Berlin Heidelberg 2011 



1 Deterministic Analysis of Observations 



1,1 Motivation 

Modeling Problems. Mathematical models can be most helpful for improving 
our understanding and dealing with relevant problems. Here are some examples: 

• Human beings were always interested in motions of celestial objects such as 
stars, planets, comets, and galaxies - attempting to understand the reasons for 
the motion and paths of celestial objects. Such an understanding was helpful for 
agriculture, navigation, the making of calendars, and even astrology. Nowadays, 
astronomy (or astrophysics) is used to investigate the formation and develop- 
ment of the universe. Such studies may provide answers to important questions, 
e.g., regarding the existence of other intelligent life, the development of the 
solar system, and the ultimate fate of the universe. 

• Energy supply is nowadays a prerequisite for maintaining the living conditions 
that we appreciate. Most people have made the stressful experience of a power 
outage: we are grateful when the light is back, refrigerator and computer work- 
ing again. More importantly, available energy is the condition for many techni- 
cal developments. The energy consumption does steadily grow, and we need 
quantitative knowledge of this development, this means a mathematical model 
for the energy consumption, to understand future needs. 

• There are also many daily situations for which we will find it helpful to have a 
simple quantitative understanding of things. Suppose that you are driving your 
car. Given a certain velocity, what is an appropriate distance to the car in front 
of you such that you are able to stop safely? This distance will change with the 
car velocity, so it needs a simple formula for this calculation that you can easily 
use. 

• Global warming is becoming a serious problem. An increase of the global tem- 
perature may cause glacial melting, Arctic shrinkage, and a worldwide sea level 
rise. Changes in the amount and pattern of precipitation may result in flooding 
and drought. There may be changes in the frequency and intensity of extreme 
weather events. Thus, an understanding of global warming is clearly relevant. In 
particular, we need mathematical models that explain human impacts on the 
global warming (e.g., as a consequence of greenhouse gas emissions). 

• Other relevant mathematical modeling tasks arise from technical developments. 
The optimal design of technical processes often requires the simulation of 
complex processes that can hardly be studied otherwise. This concerns, e.g., the 
optimization of flow and chemical reactions in chemical reactors, and the flow 
around an aircraft or wind turbine. The ability to perform accurate oil reservoir 
simulations or CO2 sequestration is a requirement, for example, to optimize the 
acquisition of natural resources. 



1.2 Data Transformations: Linear Models 
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Fig. 1.1. An illustration of global warming, (a) The HadCRUT3 global tempera- 
ture anomaly data (consisting of annual differences from 1961-90 normals) are 
given in °C (Rayner et al. 2003, Brohan et al. 2006); (b) the Mauna Loa data (Tans 
2008) of atmospheric COj concentrations are given in ppmv. 

• Another example that is clearly relevant to daily life is given by the weather 

prediction. It is helpful to know the probability for significant temperature 

changes, rain or snow, and we need knowledge about the developments of 

blizzards, hurricanes, and tornadoes to be prepared for their consequences. Such 

weather predictions have to be performed by means of numerical simulations, 

and the basis for such simulations is given by mathematical models. 

Problems Considered. The question of how it is possible to develop models 

for the first four problems described in the preceding paragraph will be addressed 

in the following sections of this chapter. The last two problems require the use of 

very advanced techniques that cannot be presented in this book. A typical problem 

considered here is illustrated in Fig. 1.1 that illustrates the global warming in 

terms of the increase of the atmospheric COi concentrations and the related 

increase of the global temperature anomaly T. A quantitative understanding of this 

relevant problem, including the explanation of the reason for the observed global 

warming, requires the development of mathematical models for these data trends 

and the analysis of the relation of these trends. A solution to this question will be 

presented in Sect. 1.5. 



1,2 Data Transformations: Linear Models 



First, let us devote ourselves to the simplest modeling approach: the application 
of linear functions. It is often the case that data considered do not follow a linear 
function. However, data transformations sometimes allow the use of linear models 
for redefined data. Several examples for this approach will be given in the 
following. Other examples will be considered in Sects. 1.4 and 1.5. 
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Table 1.1 The U.S. energy consumption C (in 
10'^ Btu) in time t (U.S. Dept. of Energy 2008). 



1.2.1 Energy Consumption 



The U.S. Department of Energy (2008) published data on the U.S. energy con- 
sumption in its Annual Energy Report 2007. The data are given from 1950 to 2005 
in Table 1.1 and shown in Fig. 1.2. Let us develop a model for these data. 

Linear Energy Consumption Model. The data presented in Fig. 1.2 support 
the use of a linear function for the modeling of the energy consumption. A linear 
model for the energy consumption C that passes any given data points (?i, Ci) and 
(?2, Cj) can be written as 



C = Q^^+C,^^ 



t, -t. 



tn-t, 



(1.1) 



The validity of this expression can be proven by considering C at t^ and ?2- This 
shows that C(?i) = Ci and C(?2) = Cj, as required. To represent the overall data 
trend, it appears to be reasonable to use a linear fUnction that passes the points 
(1950, 34.6) and (1995, 91). The use of these data in Eq. (1.1) leads to 



C = 34.6 '-'''' .91- '-'''' 
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Fig. 1.2. U.S. energy consumption C (in 10'^ Btu) in time t. (a) The data from 
Table 1.1 are shown as dots, and the line represents the linear model (1.3); (b) 
shows the relative error e (in %) of the model (Eq. 1.3). 
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A convenient way to write this formula is to reformulate both contributions by 
referring to t - 1995. Thus, Eq. (1.2) becomes 

C = 91 + 1.25(^-1995). (1.3) 

This writing of the model for the energy consumption reveals the value C = 9 1 at 
the time t = 1995 and the rate of the consumption increase (the slope 1.25). 

Model Evaluation. Figure 1.2 shows the good performance of this model. The 
trend of the energy consumption is well represented, and the model function 
agrees relatively well with the data points. The model performance can be seen in 
more detail by calculating the relative error. Given any set of data points (xi, jv,), 
(x2, yj), ■ ■ ■ (x„, j„), the error e related to each data point can be defined by 

e = -^ / ,, . (1.4) 

(mod) ^ ^ 

y 

Here, j*"""'' refers to the value predicted by the model. This value is considered to 
be the true value. Obviously, this view depends on the performance of the model, 
this means a model that clearly disagrees with any observed data trend cannot be 
considered to provide a true value. The symbol y denotes the observed data point. 
This value is considered to be affected by randomness. The relative error related 
to the use of the model (1.3), y ^ Cm our case, is shown in Fig. 1.2b. This figure 
shows that the maximum relative error of Eq. (1.3) is given by 13.5% at ^ = 1970. 
Such an error means that Eq. (1.3) represents a reasonable model for the energy 
consumption for the period considered. It is worth noting that the magnitude of the 
relative error is smaller than 3% for t > 1985. Such error values are typical for a 
good model. An attractive feature of the model given by Eq. (1.3) is its sim- 
plicity, which enables an easy understanding: the energy consumption increases 
linearly in time t with an increase rate of 1.25. The model is certainly useful. It can 
be used, for example, for making reasonable predictions of the energy con- 
sumption for the next decade. 



1.2.2 Kepler's Third Law 

Let us consider another example. The German mathematician and astronomer 
Johannes Kepler (1571-1630) analyzed over 20 years the motion of planets on the 
basis of the astronomical observations of the Danish astronomer Tycho Brahe. By 
1609 Kepler had formulated his first two laws of planetary motion: (Kl) a planet 
revolves around the Sun in an elliptical orbit with the Sun at one focus, and (K2) 
the line joining the Sun to a planet sweeps out equal areas A{t) in equal times (i.e., 
dAI dt is constant, see the illustration in Fig. 1.3). Kepler then spent many years to 
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planet 




planet 




(a) Kepler's First Law 



(b) Kepler's Second Law 



Fig. 1.3. An illustration 
of Kepler's First Law 
and Second Law. 



address the question of what determines the orbital period Tp (the number of Earth 
years to orbit the Sun with respect to background stars) of the revolution of a 
planet. The latter question is relevant to see, for example, whether there are laws 
that govern the planetary motion. 

Linearized Model. Let us use the data in Table 1 .2 to address the same ques- 
tion: how does Tp depend on the mean distance r from the Sun (the mean distance 
is the sum of the maximal and minimal distance divided by two)? Figure 1.4a 
shows that a linear model clearly disagrees with the data trend: the data increase 
faster than a linear function. Figure 1.4b shows that a quadratic function (which 
can be found by adopting the approach presented in Sect. 1.3) also disagrees with 
the data trend: this function increases faster than the data. The behavior of linear 
and quadratic models indicates that Tp can be a power function of r given by 



-Ar 



(1.5) 



Here, A and B are any constants. The best way to test the validity of this assump- 
tion is to consider a corresponding linear relation between In Tp and In r, which 
follows from taking the natural logarithm of Eq. (1.5), 

\nTp=\nA + B\rLr. (1.6) 

To prove the validity of this assumption we have to plot In Tp against In r. Indeed, 
Fig. 1.5a provides evidence for the suitability of such a linear relation. By using 
the two-point formula (1 . 1) we find (the second and the seventh point are used) 



Inn =2.8490 + 1.4996 In r. 



(1.7) 



Planet 


r 


Tp 


Mercury 


0.0579 


0.2408 


Venus 


0.1082 


0.6152 


Earth 


0.1496 


1.0000 


Mars 


0.2280 


1.8808 


Jupiter 


0.7785 


11.8618 


Saturn 


1.4335 


29.4566 


Uranus 


2.8718 


84.0107 


Neptune 


4.4948 


164.7858 



Table 1.2 Orbital periods and mean distances of 
planets from the Sun (World Almanac 2010). Here, 
;■ is the mean distance from the Sun (which is equal 
to the major axis of the ellipse) in 10' km. Tp is the 
period in earth years a = 365.256 days, this means 
Tp gives the number of Earth years to orbit the Sun 
with respect to background stars. 
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Fig. 1.4. The orbital period Tp of planets in dependence on the mean distance r 
from the Sun. The data from Table 1.2 (World Almanac 2010) are given by dots. 
The lines in (a) and (b) show linear and quadratic functions for Tp, respectively. 



By taking both sides of this relation as exponents of an exponential function, the 
dependence of Tp on r can be written as 



4n' 



(1.8) 



This is Kepler's Third Law (K3): the square of the period of revolution of a planet 
is proportional to the cube of the major axis of its orbit. Here, Gg = 0.1324 
(10' km)^ / a^ is the standard gravitational parameter. The constant Gg can also be 
written as Gg = 1.3291 lO'" mVs', where 1 km = 10' m and a = 31,558,118.4 s. 
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Fig. 1.5. The orbital period Tp of planets in 
dependence on the mean distance r from the 
Sun. The dots present the data of Table 1.2 
(World Almanac 2010). The line in (a) shows 
In Tp according to Eq. (1.7). Eq. (1.8) is com- 
pared with the corresponding data in (b). The 
relative error e (in %) related to the use of Eq. 
(1.8) is shown in c). 
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V 


D 


u 


D 


10 


27 


60 


303 


15 


44 


65 


344 


20 


63 


70 


387 


25 


85 


75 


433 


30 


109 


80 


481 


35 


135 


85 


531 


40 


164 


90 


584 


45 


195 


95 


639 


50 


229 


100 


696 


55 


265 







Table 1.3 The characteristic total stopping dis- 
tance D (in feet) related to the automobile's 
velocity u (in mph) according to the Code of 
Virginia (Jemigan & Kodaman 2001). 



Model Evaluation. Figure 1.5b shows that Eq. (1.8) agrees very well with the 
data: there is no observable difference between them. This fact indicates that 
Eq. (1.8) represents a very accurate model. Evidence for this view is provided by 
Fig. 1.5c that shows the relative error e in % related to the use of Eq. (1.8). As 
may be seen, the relative error is very small (the magnitude is smaller than 0.6%). 
It should be noted that there is no unique way to calculate the parameter Gg and 
the power 1.5 of r in Eq. (1.8). Very little modifications of these values, e.g., a 
power 1 .4996, would imply a model with the same accuracy. However, the power 
will be always found very close to 1.5. Thus, the value 1.5 is the most appropriate 
value because it indicates a deep theoretical relationship between Tp and r. The 
value Gg = 1.3291 x 10^" m^ / s' provides the best agreement between the model 
and data. The simplicity of Eq. (1.8) is very helpful for the understanding. The 
observation that the orbital period Tp is fully controlled by the mean distance r 
from the Sun means that Tp is independent of properties (e.g., of the mass) of 
planets. This feature indicates the possible existence of a universal law that 
governs the planetary motion. Indeed, it will be shown in Chap. 3 that Newton's 
Law of Gravitation can be derived on the basis of Kepler's Third Law. 



1.2.3 Vehicular Stopping Distance 

Let us consider a problem from daily life: the vehicular stopping distance. Say 
you drive your car at a certain velocity v, and then, you need to come to a full 
stop. The stopping distance D is the distance that you will still drive after trying to 
stop. Table 1.3 shows data for the dependence of D on v according to the current 
Code of Virginia (Jemigan & Kodaman 2001). Knowledge about the stopping 
distance D is relevant to keep a good distance to the car in front of you. How is it 
possible to model the dependence of D on u? In particular, how can we derive a 
formula for D(v) that we can easily use? 



1.2 Data Transformations: Linear Models 




800- 


1 1 1 1 i__. 


600- 


/ - 


DiOO- 


/ 


200- 
0- 


J^ (b) - 



20 iO 60 80 100 120 
V 



20 iO 



80 100 120 




Fig. 1.6. The total stopping distance D (in feet) 
in dependence on a car's velocity v (in mph). 
The dots show data from Table 1.3. (a) D/i; is 
compared to the function 2.2 + u /21 (line); (b) 
the perfonnance of Eq. (1.9); (c) relative error 
e (in %) related to the use of Eq. (1.9). 



Linearized Model. A reasonable approach is to think about the suitability of a 
linear function. However, Fig. 1.6 shows that a linear function cannot represent an 
accurate model. A way to see the deviations from a linear function is to plot D/v 
against v. This approach involves the assumption that D is proportional to v. This 
view is correct because D should be equal to zero for u = 0. A plot of D / v is 
shown in Fig. 1.6a. Obviously, a linear function for Z) / u represents a very good 
model. Instead of using D = (2.2 + v /21) v, which follows from the two-point 
formula (1.1) combined with the points (15, 44/15) and (95, 639/95), the function 



D=\ 2 + 



20 



(1.9) 



will be applied here. The main reason for doing this is that we are interested in a 
model that can be easily applied for the calculation of Z). 

Model Evaluation. Figure 1.6b shows that Eq. (1.9) represents a very good 
model. A closer look reveals the following: The trend of the data is well repre- 
sented, and there is a good agreement between the model and data points. The 
accuracy of Eq. (1.9) can be evaluated by considering its relative error e shown in 
Fig. 1.6c. We see that the error magnitude is smaller than 1% for u > 60 mph. For 
smaller velocities there are error magnitudes of up to 8% (at u = 10 mph). How- 
ever, it is more important to assess the stopping distance at higher velocities. 
Thus, the model accuracy is excellent. The model is simple, which is helpful for 
using it for the calculation of stopping distances including velocities which are 
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higher than the range of u considered in Fig. 1.6. Is the model understandable? We 
may expect that D depends, basically, linearly on v. Why do we observe a 
quadratic contribution v' in Eq. (1.9)? The stopping distance D must depend on 
the kinetic energy mv^ 1 2 of the car {m is the car's mass): the higher the kinetic 
energy the longer it will take it to stop. Therefore, a quadratic dependence of D on 
V is reasonable. 



1,3 Model Development: Polynomial Models 

The approach presented in Sect. 1.2 is extremely helpful because of its 
simplicity, but it has a rather limited range of applicability: most models cannot be 
developed in terms of linear functions. To deal with this problem we will consider 
polynomial models. Their use represents a flexible strategy for the development of 
relatively simple models that can represent a variety of trends. 



1.3.1 The Lagrangian Form of Polynomials 

Linear Polynomials. First, we consider linear polynomials, i.e., a function 
y = aQ + aiX. Here, x is the independent variable, and y is the dependent variable 
(any variable for which we need a model). The function jv = aQ + UiX involves two 
unknown parameters: ao^nd flp The parameters can be calculated by the constraint 
that y{x) passes through two points (x,, j,) and (Xj, y-^, which we consider to be 
given by observations. The requirement that y{x) passes through these two points 
implies the following two relations, 

y^-a^ + a^x^, y2-aQ+a^x^. (1-10) 

These relations can be used to calculate the parameters a^ and a^my ^ aa + a^x. 
This calculation provides j = /'i(x), where 

PAx)-y,^^^ + y,^^^. (1.11) 

Here, Pi{x) refers to a polynomial of first order. The term polynomial refers to a 
function that consists of the sum or difference of integer powers of x multiplied 
with any constants. The subscript one of /'i(x) refers to a polynomial of first order. 
It is assumed that the positions x^ and X2 are different to avoid division by zero. 
How is it possible to prove the claim that y = Pi(x) passes through the two points 
considered? The simplest way is to consider Pi{x) at xi and Xi- In this way, one 
finds the identities X^i) = ji and ^(xj) ^yi, as required. 
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Quadratic Polynomials. Second, we consider quadratic polynomials, this 
means a function j ^ aQ + a^ x + a2X^. This function >'(x) is completely determined 
by the constraint that y{x) passes through three given points (xi, >>[), (x2, J2) and 
(xj, jj). The latter constraint provides the relations 

2 2 2 

>>, = flg + fllX, + flj^l ) y2 = ao~^''l^2~^^2^2 ^ JVj = Go + fljXj + fljXj (1-12) 

for the calculation of the parameters Oq, Oi, and 02- The function j(x) obtained in 
consistency with these constraints can be written y = P2(x), where 

(X-X3)(X-X3) ^ (X-Xi)(x-X3) ^ (X-Xi)(x-X2) 



(1.13) 
The positions Xi, X2, and X3 are assumed to be different. The validity of this 
expression can be proven by considering PjC-*^) at Xj, X2, and X3, which provides the 

identities X^i) = Jbjfe) =J2, andjfe) =J3- 

Lagrangian Form of Polynomials. Third, we consider the general case that we 
have any polynomial of order n. To define such a polynomial uniquely, we assume 
that observations are given at « + 1 points, this means (xq, Jo), (xi, ji), ••• (x,„ y„). 
The reason for using « + 1 points for the definition of a polynomial of order n can 
be seen by considering again linear and quadratic polynomials: we need two 
points to define a linear function, and we need three points to define a quadratic 
function. The way to construct a polynomial y = P„{x) becomes clear by having a 
closer look at the structure of linear and quadratic functions. Let us consider Eq. 
(1.13) for a quadratic polynomial. For every j, (/ = 1, 3), the numerators involve 
products of differences between x and all the other positions except the difference 
X - X,. The denominators are equal to the numerators except that x is replaced by 
X,. Hence, a polynomial j = P„(x) of order n is given by 

P„(^) = yoLo(x) + jiA W + ■ --y^LA^l (i-i4) 

where the functions it(x) are defined by {k = 0, n) 

L{x)= n ^ ~ ^'' - (^ - ^0 )(^ - ^1 ) • • - (^' - ^t-i )(^' - ^M ) ■ ■ ■ (^' - ^'„ ) 

/=0, i*k Xf. — X- (Xj, — Xq j(Xj, — Xj j ■ ■ • {Xj, — X,,_j J(Xj. — Xj,^[ )■■■ (X^ — X^^ ) 

(1.15) 
Here, it is assumed that the positions xj, X2, ■■■' x„ are different to avoid division 
by zero. The correctness of this formula can be seen by considering P„Qc) at Xg, xi, 
■■■: x„. This shows that the polynomial y = P„{x) passes through all the observa- 
tions (xq, Jo), (xi, ji), ■■■ (x„, j„). The polynomial formula (1.14) is called the 
Lagrangian form of polynomials. The settings « = 1 and n = 2 reveal that the 
polynomial (1.14) generalizes the expressions for linear and quadratic functions, 
respectively. 
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Table 1.4 Data sets used for the illustration of properties of exact polynomials. 



X 


ia)y 


(b)y 


(c)y 


id)y 


ie)y 


{fly 


1 


1.0001 


1.001 


1.01 


1.05 


1.00 


1.00 


2 


1.9998 


1.998 


1.98 


1.90 


2.00 


2.00 


3 


3.0003 


3.003 


3.03 


3.15 


3.00 


3.00 


4 


3.9996 


3.996 


3.96 


3.80 


4.04 


4.80 


5 


5.0005 


5.005 


5.05 


5.25 


5.00 


5.00 


6 


5.9994 


5.994 


5.94 


5.70 


6.00 


6.00 


7 


7.0007 


7.007 


7.07 


7.35 


7.00 


7.00 


8 


7.9992 


7.992 


7.92 


7.60 


8.00 


8.00 



1.3.2 Properties of Polynomials 

Exact Polynomial Models. Polynomial models have many advantages. Poly- 
nomials can be easily differentiated and integrated. Analytical derivatives and 
integrals that are obtained in this way are helpful to find, for example, minimum 
and maximum values of variables or integral values like work and probability. For 
a given number of observations it is always possible to construct a polynomial that 
passes exactly through all the data points (« + 1 data points define a polynomial of 
order n). Such a polynomial will be called here an exact polynomial model. 
However, it turns out that models obtained in this way are often not very useful, 
the reason being that observations are always affected by errors, and such errors 
may imply a poor performance of exact polynomial models. Let us have a closer 
look at this problem. 

Erroneous Data. Data used for the development of models may be affected by 
a variety of errors. It is often the case that the conditions for experiments are 
affected by little changes. For temperature measurements in the atmosphere, for 
example, one does never find exactly the same conditions. Such little changes of 
conditions for experiments will produce noisy observations. The way in which 
data are recorded does also induce errors. Every measurement method has a 
certain error: there is usually no way to perform absolutely correct measurements. 
With regard to measurements of human populations, it is often only possible to 
obtain good estimates for the real numbers (e.g., for the number of residents of 
countries or cities). Another source of errors is given by the fact that observations 
(e.g., temperature measurements) have to be presented numerically with a certain 
number of digits. Round-off errors that are introduced in this way will also add 
errors to data sets. Occasionally, single measurements are simply incorrect 
because of a variety of possible recording problems. 
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Fig. 1.7. Exact polynomial models of 7* order for the six cases of data points 
given in Table 1.4. The data points are given by dots. The polynomial models are 
given by lines. All the curves pass exactly through the data points. The arrows in 
(e) and (f) indicate the position of incorrect values. 



Effect of Erroneous Data. Let us consider the data presented in Table 1 .4 to 
illustrate the effect of erroneous data on exact polynomial models. The cases (a) 
and (b) present noisy y data with a noise of 0.01% and 0.1%, respectively. These 
cases illustrate the influence of round-off errors. The cases (c) and (d) present 
noisy y data with a noise of !%> and 5%i, respectively. Such data may result from 
changing conditions for experiments or the inaccuracy of measurement methods. 
The cases (e) and (f) involve incorrect y values at x = 4 (errors of 1%) and 20%), 
respectively). The implications of these data sets are illustrated in Fig. 1.7 that 
shows exact polynomial models of 7* order for these six cases. These polynomials 
can be obtained by specifying the corresponding Lagrangian form of polynomials 
(1.14). Figures 1.7a and 1.7b reveal that even very minor noise effects of 0.01%o 
and 0.1%) may have significant effects on exact polynomial models: such 
polynomials are very sensitive to small changes in the data. The trend of data is 
given by a linear function y = x. Instead, the polynomials show severe oscillations 
after the interval for which data are given. Obviously, the use of such models to 
extrapolate data trends may be completely wrong. Figures 1.7c and 1.7d show that 
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Fig. 1.8. Reduced polynomial models of first order for the cases (d), (e), and (f) of 
Table 1.4. The data are given by dots, and polynomials are given by solid lines. 
The polynomials are defined by the constraint to pass through the first and last 
data point. The dashed line in (a) shows a polynomial that passes through the first 
point and the point next to the last point. 



these effects are more pronounced for higher noise intensities. In addition to the 
curve behavior in Figs. 1.7a-b, the model behavior between the data points is now 
also incorrect: see the curve for 1 < x < 2 and 7 < x < 8 in Fig. 1.7d. The curves 
present all the data points correctly, but there is no guarantee that the model is 
correct for any deviations from the data points. Figures 1.7e and 1.7f show that 
even relatively small deviations from correct recordings of data points may have 
severe implications for the performance of models. One erroneous recording with 
an error of 1% means that the model becomes unusable for the extrapolation of 
data. One erroneous recording with an error of 20% results in a model that is not 
useful at all. 

Reduced Polynomial Models. A way to avoid the extreme sensitivity of exact 
polynomials to small changes in the data is to apply low-order polynomials (e.g., 
linear, quadratic, or cubic polynomials). Such polynomials are constructed on the 
basis of a reduced number of data points. The models obtained in this way are 
called here reduced polynomial models. The properties of reduced polynomials 
are illustrated in Fig. 1.8. These functions are capable of representing the correct 
data trend. Low-order polynomials are more stable than exact polynomials: their 
use is related to a smoothing. Reduced polynomial models have, however, a major 
disadvantage because there is no unique way to construct such models. These 
models depend on the data chosen to define the polynomial considered. Figure 
1.8a illustrates that the use of different data points used to define such 
polynomials may have a significant effect on the model. 
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Fig. 1.9. The application of exact polynomials; 
(a) a Lagrangian polynomial of 1 1* order for the 
U.S. energy consumption C, (b) a polynomial of 
7"" order for the orbital period Tp, and (c) a poly- 
nomial of 18* order for the total stopping dis- 
tance D. The data points are given by dots. 



1.3.3 Polynomial Models of Observations 



Let us reconsider the examples discussed in Sect. 1.2 to illustrate the 
performance of polynomials regarding the modeling of real data. 

Exact Polynomial Models. Figure 1.9a presents a Lagrangian polynomial of 
1 1* order as a model for the U.S. energy consumption C. This model provides an 
exact curve through the 12 data points considered (the data between 1950 and 
2005). The model shows severe oscillations between the data points. The predicted 
trend after 2005 is clearly in disagreement with the data trend. Thus, this approach 
does not work at all. Figure 1.9b shows the disadvantages of exact polynomials for 
the modeling of the orbital period Tp. The polynomial of 7* order passes all the 
data points, but its behavior is incorrect for r > 2.9. Figure 1.9c shows an exact 
polynomial of 18"" order as a model for the stopping distance D. This polynomial 
provides an accurate model except for the behavior of Z) close to the end points of 
the interval considered (see the curve between ;; = 95 mph and v = 100 mph). The 
oscillations observed indicate again the disadvantages related to the use of an 
exact polynomial model. Such a model is useless for the prediction of Z) at veloci- 
ties V > 100 mph. Apart from that, knowledge of a polynomial of 18"' order will 
not help a lot to find an appropriate distance to the car in front of you: the use of 
such a polynomial is not easy! 
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Table 1.5. The development of the U.S. population in time from 1790-2050 according to the 
Decennial Censuses, U.S. Census Bureau, U.S. Dept. of Commerce (World Almanac 2010). The 
population P is measured in 10' and t refers to the year. The values for / > 2000 are projections 
(Population Division, U.S. Census Bureau, NP2008-T1, August 14, 2008). 



t 


P 


t 


P 


t 


P 


1790 


0.0039 


1880 


0.0502 


1970 


0.2033 


1800 


0.0053 


1890 


0.0630 


1980 


0.2265 


1810 


0.0072 


1900 


0.0762 


1990 


0.2487 


1820 


0.0096 


1910 


0.0922 


2000 


0.2814 


1830 


0.0129 


1920 


0.1060 


2010 


0.3102 


1840 


0.0171 


1930 


0.1232 


2020 


0.3414 


1850 


0.0232 


1940 


0.1321 


2030 


0.3735 


1860 


0.0314 


1950 


0.1513 


2040 


0.4057 


1870 


0.0398 


1960 


0.1793 


2050 


0.4390 



Reduced Polynomial Models. The use of reduced polynomial models for the 
modeling of the U.S. energy consumption C, the orbital period Tp, and the total 
stopping distance D shows different features. After rewriting the orbital period 
and stopping distance data, it was shown in Sect. 1.2 that all the three problems 
can be successfully solved on the basis of linear functions. Further examples for 
the usefulness of reduced polynomial models will be discussed in Sects. 1.4 and 
1.5. Thus, the application of reduced polynomial models is in general much more 
helpful than the use of exact polynomial models. 



1.4 Model Evaluation: Population Modeling 

Two approaches for creating a model were considered so far: the development 
of linear models (for transformed data if required) in Sect. 1.2 and the use of 
polynomial models in Sect. 1.3. Next, let us consider the relevant question of how 
a model can be evaluated. This question will be addressed by considering the 
development of the U.S. population in time according to the Decennial Censuses, 
U.S. Census Bureau, U.S. Dept. of Commerce (World Almanac 2010). The data 
are given in Table 1.5. First, we will address the modeling of the U.S. population 
in terms of linear models for redefined data. This approach is driven by concepts 
for the modeling of population dynamics that will be described in detail in 
Sect. 7.4 (here we use these concepts simply as functions that can be transformed 
to linear models for redefined variables). Second, we will consider the application 
of polynomial models. The second approach is driven by data - we try to find an 
appropriate model as an interpolation and extrapolation of available data. Third, 
we evaluate the suitability of these two modeling approaches for the development 
of the U.S. population. 
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Fig. 1.10. An exponential model for the develop- 
ment of the U.S. population P (in 10') in time t. 
The dots present the Table 1.5 data. The lines in 
(a) and (b) present the models (1.18) and (1.19). 
The relative error of Eq. (1.19) is shown in c). 



1.4.1 Linearized Conceptual Models 



Several concepts for the modeling of population dynamics will be discussed in 
Sects. 7.4 and 7.5. Here, we will focus on the application of two models that can 
be written as linear models for redefined variables: the exponential model (7.82), 
which is called the Malthusian Law, and the logistic model (7.102). 

Exponential Growth. The Malthusian Law (7.82) represents the assumption 
that populations grow exponentially: 



P = P,e' 



C-'o) 



(1.16) 



Here, Pq is the initial value of P at t^,, and r is the growth rate. Equation (7.82) is 
used here in a slightly modified manner because of the consideration of a nonzero 
initial time ?o- The most convenient way to look at the Malthusian Law is to write 
it as a linear relation between InP and t. 



\nP = \nP^ +r(t-tg). 



(1.17) 



The natural logarithm of the population data is shown as a function of t in 
Fig. 1.10a to see the suitability of this assumption. Hence, In P can be well 
described by a linear function of t for the population data from 1790 to 1890. The 
parameters of the linear function are found by using the two-point formula (1.11). 
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Fig. 1.11. A logarithmic model for the develop- 
ment of the U.S. population P (in 10') in time t. 
The dots present the Table 1.5 data. The lines in 
(a) and (b) present the models (1.22) and (1.23). 
The relative error of Eq. (1.23) is shown in c). 



The linear function obtained in this way is given by 

?-1790 
lnP = hi0.0039 + - 



35 



(1.18) 



Here, the initial value P(1790) = 0.0039 is used according to the data in Table 1.5, 
and the growth rate /' = 1/35 provides the correct increase of population data. The 
latter model implies the exponential model 



P = 0.0039e 



((-1790)/35 



(1.19) 



Figure 1.10b shows a good agreement with the model and data. Fig. 1.10c shows 
that the magnitude of the relative model error is smaller than 9%. 

Logistic Growth. Another modeling concept is to assume a logistic growth. 
The logistic model can be written in several ways (see Sect. 7.4.2). We will use 
the centered logistic model of Eq. (7. 104), 



p= p 



1 + - 



1- 



-(,.)/r A 



\ + e- 



2P 



1 + e 



-('-', )/r ■ 



(1.20) 



Here, P^ = P(tc), t^ is the critical point at which dPI dt has a maximum, and r refers 
to a characteristic time for the transition to the equilibrium state. In particular, we 
use Eq. (7. 104) here in a simplified version by neglecting a constant P_^, that may 
be added to P (there is no need to involve such an additional constant P_„ here). 
The best way to prove the suitability of the population model (1.20) is again to 
look at the linear relationship between ln(2P^ I P - \) and t that is implied by 
Eq. (1.20), 
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ln{2PJP-l) = -'—^. (1.21) 

T 

To compare the logistic model with the exponential model we consider again the 
population data from 1790 to 1890. Figure 1.11a shows that ln(0.12/P- 1) can be 
modeled by a linear function: 

ln(0.12/P-l)=-^^i^. (1.22) 

The rewriting of this relation results in the logarithmic population growth model 

" '■'' (1.23) 



1 + ^-{l-l«90)/29 ■ 

The parameters of this model are chosen such thatP(1890) = 0.06 according to the 
data in Table 1.5, and the growth rate 1/29 provides the correct increase rate. 
Figure 1.11b shows good agreement between the model and the data. This view is 
supported by Fig. 1.11c that reveals the magnitude of the relative model error to 
be smaller than 5%. 



1.4.2 Polynomial Models 

An alternative approach for the model development is the use of polynomials 
discussed in Sect. 1.3. In agreement with the discussion of the properties of exact 
polynomial models, it turns out that the use of a polynomial of 26"' order that 
provides an exact curve through the 27 data points given in Table 1.5 is pointless: 
a reasonable model cannot be obtained in this way. Thus, we have to use reduced 
polynomial models. Here, we will use linear and quadratic models. 

Quadratic Polynomial. First, we will consider the suitability of a quadratic 
polynomial. According to Eq. (1.13), the quadratic polynomial is given by 

^_^ (t-t,)it-t,) ^^ {t-t,)it-t,) ^^ (t-t,)it-t,) 
\t,-t^){t,-t,) \t,-t^){t^-t,) \h-t,){t,-t,)' 

The population data at 1790, 1890, and 2040 are taken as the three data points 
required for the use of this formula. The performance of the quadratic polynomial 
is illustrated in Fig. 1.12a. This polynomial describes the population data over the 
entire data range considered. The model error is given in Fig. 1.12b. The error 
values have a peak of -32% ait= 1810. However, the population values are very 
small in this range so that such errors are acceptable. For ^ > 1 840, the error values 
are found to be relatively small, -3.4% < e < 8.5%. Thus, the second-order 
polynomial model represents a good model. 
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Fig. 1.12. Polynomial models for the development of the U.S. population P (in 
10') in time t. The dots represent the U.S. population data. The line in (a) shows 
the quadratic polynomial (1.24), and (b) shows the corresponding model error in 
%. The linear model (1.25) and its error are shown in (c) and (d), respectively. 

Linear Polynomial. Another approach is to focus the modeling on the data 
range that is most relevant: the population data for t > 1990. Figure 1.12 shows 
that a linear model may work for this data range. From Eq. (1.11) combined with 
the population data at 2000 and 2040 given in Table 1.5 we obtain the function 



P = 0.28 + 



?-2000 ?-1910 



322 



322 



(1.25) 



The model error is shown in Fig. 1.12d for the range t > 1990. The magnitude of 
the relative error is smaller than 1%, which means that Eq. (1.25) represents a very 
accurate model. 



1.4.3 Model Evaluation 



Four population models were presented in Sects. 1.4.1 and 1.4.2 - but which 
model is the best one? Let us consider the properties of reasonable and optimal 
models to prepare the answer to this question. 
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Table 1.6 Basic properties of reasonable and optimal mathematical models. 

Basic Properties of a Reasonable Model Basic Properties of an Optimal Model 

la. Agreement with the la. Accurate agreement 
trend of observations. with observations. 

lb. Agreement with observations over lb. Accurate agreement with all obser- 
a certain data range of one case. vations of one class of problems. 

2a. Information in addition to observa- 2a. Information in addition to observa- 
tions results from using a function. tions results from a model concept. 

2b. Interpolation and extrapolation of 2b. Valuable new insight into the 

information given by observations. mechanisms of observed processes. 



Reasonable and Optimal Models. Characteristic properties of reasonable and 
optimal models are summarized in Table 1.6. There are also other criteria like the 
simplicity of formulation, cost and ease of use, but these additional criteria are not 
really relevant to the problems considered here. A basic requirement for a model 
is given by support through observations. The agreement with observations can 
range from a reasonable agreement with the trend of observations (reasonable 
model) up to an accurate agreement (errors below 1%) with observations (optimal 
model). An essential difference between models is given by the range over which 
a model agrees with observations. A reasonable model should be applicable to a 
certain data range of one case, whereas an optimal model should be applicable to 
all observations of relatively similar cases. Models also differ by the kind of 
support for the information provided in addition to the information given by 
observations, i.e., the basis for the modeling approach. Such support can range 
from the use of a function as given for polynomial models up to the use of a 
modeling concept that explains the nature of a class of problems (optimal model). 
In the end, the most basic requirement for a model is that the model has to be 
helpful, which means a model should provide more information than given by the 
observations. Such additional information can range from the interpolation and 
extrapolation of information given by observations (reasonable model) up to a 
valuable new insight into the mechanisms of observed processes (optimal model). 

Linearized Conceptual Versus Polynomial Modeling. Let us use the criteria 
for reasonable and optimal models described in the preceding paragraph to 
evaluate the standard of linearized conceptual and polynomial models presented in 
Sects. 1.4.1 and 1.4.2, respectively. All the four population models satisfy the 
criteria for a reasonable model, but there are differences regarding the properties 
of optimal models. The advantage of linearized conceptual models is that these 
models are based on modeling concepts. The disadvantage of these two models is 
that these concepts are supported by observations only for a certain range of data 
values, and there are two models that have the same type of support. Do these 
models provide valuable new insight into the mechanism of this process? The 
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latter is not the case because there is no convincing evidence for the concepts 
represented by these two models. The advantage of polynomial models is their 
range of applicability and accuracy: we have the choice between a rather accurate 
quadratic model for the entire data range and a simple and very accurate linear 
model that describes the development for t > 1990 with a relative error below 1%. 
The disadvantage of polynomial models is that they do not present an explanation 
for the mechanism of this process: such models describe what we see anyway. 
Thus, no single out of the four population models considered represents an opti- 
mal model. How is it possible to develop an optimal model? Polynomial models 
do not have the potential to represent an optimal model (polynomial models are 
case dependent so that their model parameters change for every case considered). 
Thus, the only way is to refine the modeling concepts considered above to obtain a 
model that agrees well with observations over the entire data range. 



1,5 The Advantage of Modeling: Global Warming Modeling 

So far, we discussed two approaches for the modeling of given observations, 
and we compared characteristic features of the modeling approaches considered. 
These modeling approaches provide functions that agree in some way with the 
observed data. Such functions are helpful because they extend the information 
given by the observations via the interpolation and extrapolation of data. On the 
other hand, such functions do not directly provide valuable new insight into the 
nature of observed trends. The latter may require combinations of conclusions 
obtained from models. Such a combination of analytical results that are derived 
from observations will be described here in conjunction with the discussion of a 
relevant problem: the modeling of global warming. 



1.5.1 The Greenhouse Effect 

Greenhouse Effect. The greenhouse effect is the process by which radiative 
energy (heat) leaving the Earth's surface is absorbed by certain atmospheric gases 
- water vapor (HjO) is the most abundant greenhouse gas, followed by carbon 
dioxide (CO2) and other trace gases - called greenhouse gases. The greenhouse 
gases re-radiate this energy in all directions, including back down towards the 
Earth's surface. Without the natural greenhouse effect, the Earth's temperature 
would be about -18°C instead of its present +14°C. Thus, the greenhouse effect 
helps to regulate the temperature of our planet: it is essential for life on Earth. 
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Table 1.7 HadCRUT3 global temperature anomaly data 
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Table 1.8 CO2 concentrations in ppmv from 1959-2007. The data were measured at the Mauna 
Loa Observatory in Hawaii (ftp://ftp.cmdl.noaa.gov/ccg/co2/trends/co2_annmean_ mlo.txt). 
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Global Warming. Observations reveal a significant increase of Earth's global 
surface temperature after about 1980. This development is very dangerous. An 
increase in global temperature may cause glacial retreat, Arctic shrinkage, and 
worldwide sea level rise. Changes in the amount and pattern of precipitation may 
result in flooding and drought. There also may be changes in the frequency and 
intensity of extreme weather events. The reason for the current global warming is 
given by human activity leading to an enhancement of the greenhouse effect by 
the emission of greenhouse gases (mostly CO2 from combustion of coal, oil, and 
gas; plus a few other trace gases) through fossil fuel combustion and deforesta- 
tion; pre-industrial levels of CO2 (prior to the start of the Industrial Revolution) 
were about 280 parts per million by volume (ppmv), and current levels are greater 
than 380 ppmv. The global concentration of CO2 in our atmosphere today far 
exceeds the natural range over the last 650,000 years of 180 to 300 ppmv. 

Temperature Increase. Evidence for the global warming of the Earth is given 
by measurements of the global temperature anomaly. The global temperature 
anomaly represents the deviation from a certain standard temperature. Usually, the 
standard temperature is based on averaged temperature values over 30 years. Here, 
we consider averaged temperatures from 1961-1990. It is relevant to note that 
these standard temperature values depend on the location of measurements. 
Table 1.7 shows global temperature anomaly data. In particular, this table shows 
the HadCRUT3 annual global surface temperature anomaly data in °C from 1850 
to 2008 reported by the UK Met Office Hadley Centre for Climate Change 
(Rayner et al. 2003, Brohan et al. 2006, http://hadobs.metoffice.com, http://www. 
metoffice.gov.ukyresearch/hadleycentre/obsdata/HadCRUT3.html). 

CO2 Emission Increase. Evidence for the increasing concentration of atmos- 
pheric CO2 is provided by the NOAA Earth System Research Laboratory (ESRL) 
data of atmospheric COj concentrations in ppmv from 1959-2007 (http://www. 
esrl.noaa.gov/gmd/ccgg/trends. Tans 2008). The data were measured (remote from 
local sources of pollution) at the Mauna Loa Observatory in Hawaii at an altitude 
of about 4 km on the peak of the Mauna Loa mountain in Hawaii (data given in 
Table 1.8). 



1.5.2 CO2 Concentration and Global Temperature Modeling 

CO2 Concentration Modeling. First, let us try to develop a model for the CO2 
concentration development. The CO2 development shown in Fig. 1.13a indicates 
relatively little deviations from a linear function. Thus, we consider the function 

CO,=CO,it,) + Sit){t-t,) (1.26) 
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Fig. 1.13. COj concentrations. Measured COj 
concentrations in ppmv (dots) are compared in 
(a) with the quadratic model (1.29) (line). The 
dots in (b) show measured data for S{t), which is 
given by Eq. (1.27). The line shows the linear 
model (1.28). The relative error of the model 
(1.29) is shown in c) in %. 



1950 1970 



as a basic model for the CO2 development, with initial values ^0 ~ 1959 and 
C02(to) = 316 ppmv. The slope function S{t) is defined by the rewritten Eq. (1.26), 



Sit): 



CO,-CO,it,) 

t-t,, 



(1.27) 



which enables the calculation of 5(0 from data. Figure 1.13a shows that S{t) can 
be modeled by the following linear function of t. 



5(0=0.7 1 + 



t-t, 
47 



(1.28) 



The combination of Eq. (1.26) with this expression for S{t) leads then to a quad- 
ratic model for the development of the CO, concentration: 



f-1959 
CO, =316 + 0.711+ , 

^ ' 47 j 



|(?-1959). 



(1.29) 



The small deviations seen in Fig. 1.13a do not affect the model performance. 
These deviations are multiplied with (?-1959), which is small in this data range. 
Figure 1.13c indicates the relative error magnitude as being smaller than 0.8%. 

Global Temperature Modeling. The modeling of the global temperature 
increase represents a rather challenging problem (see Fig. 1.14a). It is obvious that 
the global temperature data are significantly affected by randomness. To get a 
guideline for dealing with this randomness, let us consider averaged temperatures. 



26 



1 Deterministic Analysis of Observations 



0.8 
O.i 

ro.o 

-0.4 
-0.8 




T 1 1 r 

1840 1900 1960 2020 
t 



0.8 - 


1 1 

(b) 


-1 1 1 


0.4 - 




•• - 
• 


- 




• 


T 0.0 - 
-0.4- 
-0.8 - 


• 

1 1 


• 
1 1 1 



1840 1900 1960 2020 
t 




1840 1900 1960 2020 
t 



1840 1900 1960 2020 
t 



Fig. 1.14. Global temperature anomaly data: (a) temperature data without any 
averaging; (b), (c), and (d) show the temperature data averaged over 5, 10, and 20 
years, respectively. 



Such averaged temperature values are shown in Fig. 1.14, where averages over 5, 
10, and 20 years are applied, respectively. For example, the 5-year average at 
t= 1852 is obtained by adding the temperatures from 1850 to 1854 and dividing 
the sum by five. Figure 1.14d demonstrates that the amount of randomness can be 
reduced in this way. To derive a model for the global temperature anomaly we 
will use the relatively smooth 20-year averaged data shown in Fig. 1.1 4d. The data 
support the view that the temperature values are approximately constant up to 
about t = 1920. We may assume that a constant value T = -0.35 works for this 
range. For later times, the temperature values seem to increase like a power func- 
tion. So let's try to work with the model function 



-0.35 + 



?-1840 



(1.30) 



The reference to 1840 corresponds to the assumption that T = -0.35 for t values 
that are not too far from 1840. The next step is to detenxiine the model parameters 
a and b. For doing this, it is convenient to linearize Eq. (1.30), 



ln(r + 0.35) = Z>ln! 



^-1840 ^ 
a ) 



(1.31) 
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Fig. 1.15. The global temperature increase. The dots ii (a) show ln(r + 0.35) 
versus ln(? - 1840) according to the 20-year averaged observations at 1960, 1980, 
and 2000. The line presents Eq. (1.32). A comparison between 20-year averaged 
observations and Eq. (1.33) is shown in (b). Fig. (c) compares the model (1.33) 
with the non-averaged temperature anomaly data. Fig. (d) shows the relative error 
e (in %)ofthe model (1.33). 



A plot of ln(r-l- 0.35) versus ln(?- 1840) may show whether the power function 
approach applied will work. However, Fig. 1.1 4d shows that there are three data 
values that are smaller than T = -0.35, which means that the natural logarithm of 
T -1-0.35 would have negative arguments. First of all, the parameter a andfo \alues 
have to describe correctly the increase of temperature values for t > 1960. Thus, 
we will only apply the averaged data at 1960, 1980, and 2000. A corresponding 
plot is given in Fig. 1.15a. This figure also shows that the function 



ln(r-i-0.35) = 41n| 



/j-1840 
178 



(1.32) 



does well agree with the observed data. The use of a slope that is slightly higher 
than the value four applied here may result in a slightly improved agreement 
between the model and observations. However, it would be more difficult to 
illustrate the relationship between the global temperature anomaly and CO2 in this 
case (see Sect. 1.5.3). Apart from that, it is relevant to see that the 20-year 
averages are only used as guideline for the model development. According to 
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Eq. (1.32), the global temperature anomaly development is described by 

Figures 1.15b and 1.15c reveal that this model agrees reasonably well with both 
the 20-year averaged observations and the entire range of observations. 

Global Temperature Model Evaluation. Figure 1.15d shows the relative er- 
ror for the model (1.33) - the relative error is large in general. In particular, the 
maximum error magnitude is given by 6340%. A closer look at the data shows 
that huge error values of more than ±200% appear in an area where the magnitude 
of ris relatively small, -0.06 <T < 0.06 (between 1971 and 1982). For example, 
at ? = 1 977, the difference between the observed and measured temperature 
anomaly is given by 0.058°C. However, by dividing this difference by the small 
modeled temperature of r= 0.000915°C one ends up with a huge relative error: e 
= 63.4, this means 6340%). The error calculation in this range of small modeled 
temperature anomaly values is simply inappropriate. This means such relative 
error values should not be considered. The typical error values fluctuate in the 
range ±100%o. These error values pose questions about the suitability of evaluating 
the performance of models for observations that involve a significant amount of 
randomness. Such questions will be addressed in Chap. 2. 



1.5.3 The Advantage of Modeling 

Model Applications. Compared to the measured data, one advantage of the 
CO2 concentration formula (1.29) and the global temperature anomaly formula 
(1.33) is that we can use these models for making predictions. According to 
Eq. (1.29) we find a CO2 concentration of 710.8 ppmv by the end of the 2P' 
century (at t = 2100). This value is 154% above the pre-industrial concentration 
value of 280 ppmv! This formula also shows that the CO, emission per year is 
increasing. In 2000, the CO2 concentration increase rate (the difference of COt 
emissions from 2000 to 2001) was 2 ppmv yr '. In 2010, the increase rate has been 
2.3 ppmv yr '. According to Eq. (1.33), we have to expect a global temperature 
anomaly of 4.2°C by the end of the 2P' century, this means a temperature increase 
of 4.55°C compared to the temperatures in 1840! Obviously, such a temperature 
increase will imply dramatic consequences. These predictions are helpful for the 
understanding of the dimension of this problem, but they do not explicitly explain 
the relationship between the observed increase of the global temperature anomaly 
and the CO2 concentration, which is relevant to deal with this problem in an 
appropriate way. 



1.5 The Advantage of Modeling: Global Warming Modeling 



29 



T-COi Relation. To clearly see the effect of increasing CO2 concentrations on 
the temperature increase, let us combine Eqs. (1.33) and (1.29). The time t can be 
expressed as a function of Taccording to Eq. (1.33), 



? = 1840 + 178(7 + 0.35)"\ 
By relating tio Tm Eq. (1.29) we find 



(1.34) 



CO2 =316 + 0.7 



1 + 



178(7 + 0.35)"* -119^ 

47 



(l78(r + 0.35)""' -119) 



= 316 + ^178^ 
47 



'iI + f(r + o.35)--ii^V 
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(r + 0.35)' 



119 
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(1.35) 



This relation can be written as a quadratic equation: 

= f (r + 0.35)"* - — T + —{{T + 0.35)"* - 
V 178J 178V 

The solution of this equation reads 
119 47 



119 
178 



47 CO3 -316 

0.7 178' 



(1.36) 



r = -0.35 + 



-0.35 + 
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119 47 
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r 



+ J1 



0.7-47 



(C02-316)-l 



(1.37) 



The square root combined with the positive sign provides for CO2 = 316 ppmv at 
t=\959 a global temperature anomaly T = -0.35 + (119/178)"* = -0.15, which 
agrees with the consequence of Eq. (1.33). Thus, the formula for T{COt) reads 



-0.35 + 



119 47 
178 356 



1 + — (CO, -316)-1 
329V 2 I 



(1.38) 



A plot of temperature variations according to Eq. (1.38) is shown in Fig. 1.16. The 
range of CO2 values considered corresponds to the CO2 variation between 1959 
and 2010 (Eq. (1.29) predicts a value of CO^ = 390.4 ppmv at t = 2010). Figure 
1.16 reveals an almost linear increase of temperature values with the CO2 concen- 
trations. 

Linear T-COj Relation. To see the difference between the T-CO2 relation 
(1.38) and a linear function, let us derive a linear approximation to this relation. 
The first-order Taylor series expansion of Eq. (1.38) at any reference time Iq reads 

' dT ^ 
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■nt,)+ 



dCO, 
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(1.39) 
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Fig. 1.16. The temperature-COj relation. The solid 
line shows this relation according to Eq. (1.38). 
The dashed line shows the linear approximation 
(1.41). 



The derivative that appears here is given by 



dT 
dCO, 



--A{T + Q35y 
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(1.40) 
Here, Eq. (1.38) is used to replace COj by Tin order to obtain simpler expressions. 
Different linear approximations can be found in dependence on the choice of t^. 
Here, we will use ?o " 1976.9 because T{t^ = for this case. The calculation of T, 
CO2, and dTI dCOi at t^ = 1976.9 leads then to the following linear approximation 
to Eq. (1.38), 



ji(LiN) _ CO2 333.3 
120.6 



(1.41) 



Figure 1.16 shows that this linear function agrees very well with the exact relation 
(1.38): there is hardly any observable difference. This formula shows that the 
global temperature anomaly grows linearly with the CO2 concentration. Thus, the 
increase of the global temperature can be reduced by reducing the COt emissions. 
Such a clear explanation regarding the driving force of global warming cannot be 
obtained by means of observations only, which illustrates the value of modeling. 
The relevance of the numbers involved in Eq. (1.41) can be seen by relating P''"^^ 
to a nondimensional CO2 change (we have C02(1976.9) = 333.3 ppmv). 



n(LIN) 



2.76-^^' 



CO2 (1976.9) 



CO. (1976.9) 



(1.42) 



Hence, a 10% increase of the CO2 concentration compared to the level of 1976.9 
resulted in a temperature increase of 0.28°C. 



1.6 Summary 31 




Test 



y = ax + b 



Test 



Iny = \na + bx 



x + c 



: + b 



In 



y 



\ny = Ina + blnx 




1 \ = -c(x-b) 



In 



y 
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Fig. 1.17. Basic function types that can be linearized. The linearity test for the 
applicability of the each model type is shown below the pictures. 



1.6 Summary 



Let us summarize the observations made in this chapter by following the basic 
four steps of the modeling process described in the beginning. 

Data Transformations. The most efficient way to reduce modeling problems 
is to apply data transfonnations such that linear relations between redefined data 
can be considered. The use of this approach was discussed here in terms of several 
examples: the modeling of the energy consumption, planetary motion, the total 
vehicular stopping distance, the modeling of population dynamics, and the global 
warming. Figure 1.17 illustrates that a variety of data trends can be described by 
basic functions that can be linearized in terms of redefined data. For all these 
functions. Figure 1.17 also shows the linearity test (these linearity relations have 
to be satisfied in order to use the corresponding model type). The consideration of 
such linear relations is also the most convenient way for using observations for the 
calculation of optimal model parameters (which minimize the deviations between 
the model and observations: see Chap. 2). 
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Model Development. Apart from the use of linear models (if required for 
redefined data), the simplest modeling approach is the use of polynomial models. 
One way to work with polynomials is the use of exact polynomial models. This 
modeling approach appears to be very attractive because exact polynomial models 
agree exactly with any data set considered, and they are completely defined for 
any given set of data (which minimizes the modeling effort). However, it turns out 
that polynomials of higher than third order are often not helpful for the solution of 
practical questions. Such polynomials reveal an extreme sensitivity to small data 
variations, which often implies severe oscillations of the model between data 
points and a completely incorrect behavior at the end of the interval considered. 
These problems can be avoided by the use of low-order polynomials (linear, 
quadratic, or cubic functions). Their use requires, however, a careful choice of the 
data points that determine the polynomial considered. Examples for the usefulness 
of this approach were given here regarding the discussion of population dynamics. 

Model Evaluation. The next step after developing a model is its evaluation. 
Table 1.6 summarizes criteria for the evaluation of the model types considered 
here (the agreement with observed data and the information provided by models). 
The basic goal of such a model evaluation is to provide evidence that the proposed 
model fulfills the standards of a reasonable model: the model will be helpful in 
this case. Regarding the population dynamics it was shown that it is well possible 
to develop several reasonable models for any problem considered, but such 
models do often not represent optimal models. A relevant requirement for the 
development of an optimal model is the use of a concept for the model 
development. The development of model concepts will be discussed in detail in 
the following chapters. An example for an optimal model was given here by the 
highly accurate Kepler's Third Law (the model concept used in this case is the 
linearity between In Tp and Inr, which is supported by theory: see Sect. 3.3.2). 

Demonstration of the Advantage of Models. The purpose of the development 
of a mathematical model is to obtain more information about the state or the 
process considered. Usually, the development of models that have support from 
data is helpful due to the interpolation and extrapolation of data values, which is 
obtained by the model. However, models can also provide valuable insight in 
addition to the information provided by observations. An example for such a sig- 
nificant contribution of a model was given here regarding the discussion of global 
warming. We derived the linear temperature-CO, relation (1.41), which explains 
the driving mechanism of this process. Such a demonstration of the advantages of 
mathematical models does not directly follow from the development of models in 
agreement with observations: appropriate combinations of model results may be 
needed to derive conclusions that are as simple and clear as possible. 
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1,7 Exercises 

1.2.1 Consider the following data. 



X 


1 


2 


3 


4 


5 


6 


7 


y 


100 


25 


11 


6 


4 


3 


2 



a) Plot \ny versus x and Inx. Compare the data with a linear function if one 
of these plots supports the use of a linear function. 

b) Graph the table data and the model that follows from the above relation. 

c) Calculate the relative error of your model in %. 

d) At which x is the value ofy &ix = 2.5 reduced by 25%? 



Consider the following data. 










X 1 2 3 


4 


5 


6 


7 


J 2 35 150 


500 


1250 


2500 


5000 



1.2.2 



a) Plot \ny versus x and Inx. Compare the data with a linear function if one 
of these plots supports the use of a linear function. 

b) Graph the table data and the model that follows from the above relation. 

c) Calculate the relative error of your model in %. 

d) At which x is the value of >> at x = 2.5 increased by 150%? 

1.2.3 Consider the following data. 



X 


1 


2 


3 


4 


5 


6 


7 


y 


460 


280 


170 


103 


63 


38 


23 



a) Plot In J' versus x and Inx. Compare the data with a linear function if one 
of these plots supports the use of a linear function. 

b) Graph the table data and the model that follows from the above relation. 

c) Calculate the relative error of your model in %>. 

d) At which x is the value of j equal to y{x = 0) / 100? 

1.2.4 Consider the following data. Assume that the data set can be described by a 
function J = c + ax'', where a, b, and c are any constants. 



X 





1 


2 


3 


4 


y 


100 


101 


111 


140 


217 



a) Linearize y = c + a x'' and use the data to test the suitability of this 
assumption. Find a linear function for this data plot. 

b) Graph the table data and the model that follows from the above relation. 

c) Calculate the relative error of your model in %>. 

d) Find the x at which the value of j(x = 0) is doubled. 



34 1 Deterministic Analysis of Observations 



1.2.5 Consider the following data. 



X 





1 


2 


3 


4 


y 


-30 


-10 


-13 


-20 


-25 



Assume that the data set can be described by a function y = c + ax e*', 
where a, b, and c are any constants. 

a) Linearize y = c + ax e'" and use the data to test the suitability of this 
assumption. Find a linear function for this data plot. 

b) Graph the table data and the model that follows from the above relation. 

c) Calculate the relative error of your model in %. 

d) Calculate the maximum ofy{x). 

1.2.6 Consider the case where you have data that can be modeled by the function 
J = Jo + a (exp[Z'(x-Xo)] - 1). Here, a, b, and {xq, y^) are any constants. 
Explain how the data can be used to determine the constants. 

1.3.1 The U.S. Bureau of Public Roads determined the following total stopping 
distances D (in ft) depending on the velocity v (in mph) of cars. 



V 


20 


30 


40 


50 


60 


70 


80 


D 


42 


73.5 


116 


173 


248 


343 


464 



a) Use the data to plot InZ) versus v and In v. Compare the data with linear 
functions that reveal the parameters of corresponding exponential and 
power function models. 

b) Use the data to plot D/v. Compare the data with a hnear and a quadratic 
function. 

c) Plot D according to the original data in comparison to the two models 
for D/v. Calculate the relative error of the two polynomial models. 

d) Discuss the suitability of the models obtained for D. Identify one model 
that provides a formula that can be used to calculate the total stopping 
distance without using a calculator. Illustrate the use of this formula by 
three examples. 

1.3.2 Consider again the total stopping distances data given in problem 1.3.1. 

a) Use the entire data set to define an exact polynomial of sixth order. 
Graph this polynomial and the data. Comment on the suitability of this 
model. 

b) Find a way to improve the performance of this model significantly (you 
may replace one data point by other reasonable data). 

c) Use three data points to define a reasonable quadratic polynomial. 

d) Reduce the quadratic polynomial to a simple formula for D. 
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1.3.3 Consider the development of the world population in time from 1804-2050 
according to the Decennial Censuses, U.S. Census Bureau, U.S. Dept. of 
Commerce (World Almanac 2010). The population P is measured in 10' 
and t refers to the year. The last two population values are projections. 



t 


1804 


1927 


1960 


1974 


1987 


1999 


2009 2025 2050 


p 


1.0 


2.0 


3.0 


4.0 


5.0 


6.0 


6.77 7.95 9.32 



a) Use the data from 1804 to 2009 to define an exact polynomial of sixth 
order. Graph this polynomial and the data. Comment on the suitability 
of this model. 

b) Use the data from 1960 to 2009 to define an exact polynomial of fourth 
order. Graph this polynomial and the data. Comment on the suitability 
of this model. 

c) Use the data at 1960, 1987, and 2009 to define a polynomial of second 
order. Graph this polynomial and the data. Comment on the suitability 
of this model. 

d) Use the 1960 and 2009 data to define a polynomial of first order. Graph 
this polynomial and the data. Comment on the suitability of this model. 

1.3.4 Consider the following atmospheric CO2 concentrations (see Table 1.8). 



t 


1995 


1997 


1999 


2001 


2003 


2005 


2007 


C02 


360.62 


363.47 


368.14 


371.07 


375.78 


379.76 


383.71 



a) Use all the data to define an exact polynomial of sixth order. Graph this 
polynomial and the data. Comment on the suitability of this model. 

b) Use the data from 1995, 1999, 2003, and 2007 to define a polynomial of 
third order. Graph this polynomial and the data. Comment on the suita- 
bility of this model. 

c) Use the data from 1995, 1999, and 2007 to define a polynomial of sec- 
ond order. Graph this polynomial and the data. Comment on the suita- 
bility of this model. 

d) Use the 1995 and 2007 data to define a polynomial of first order. Graph 
this polynomial and the data. Comment on the suitability of this model. 

1.4.1 The following data describe the changes of a certain population P in time t 
(in days). Develop a model for the data based on the logistic model P = a I 
[\+bQ")\. 



t 


1 


2 


3 


4 


5 


6 


7 


p 


43 


58 


76 


90 


97 


106 


112 
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a) Rewrite this function as a linear relation between redefined variables. 

b) Graph the data such that the linearity assumption can be tested. Compare 
the data in this plot with a linear function to find the model parameters. 

c) Present the model obtained and graph both the model and the original 
data given in the table. Graph the relative error of the model. 

d) Use the model to predict the time at which the initial population at ? = 
is increased by a factor of 2. 

1.4.2 The following data describe the changes of a certain population P in time t 
(in days). Develop a model for the data based on the logistic model P = a I 
[\+bQ")\. 

? 1 2 3 4 5 6 7 

P 96 92 83 69 50 31 17 8 

a) Rewrite this function as a linear relation between redefined variables. 

b) Graph the data such that the linearity assumption can be tested. Compare 
the data in this plot with a linear function to find the model parameters. 

c) Present the model obtained and graph both the model and the original 
data given in the table. Graph the relative error of the model. 

d) Determine the time at which P= \. 

1.4.3 Consider again the world population data given in problem 1.3.2. Assume 
that the population P can be described by the function 



\ + bi 



- + d. 



where a, h, c, and d are any constants. For a certain time period before 
1804, the population can be approximated by a constant value P = I. 
Assume that the population density levels off finally at a value of P = 11. 

a) Rewrite the model for P as a linear relation between redefined variables. 

b) Graph the data such that the linearity assumption can be tested. Compare 
the data in this plot with a linear function to find the model parameters. 

c) Present the model obtained and graph both the model and the original 
data given in the table. Graph the relative error of the model. 

d) Find the time at which the population change dPI dt has a maximum. 

1.5.1 Consider the following data for the global temperature anomaly T and 
atmospheric CO2 concentration (see also Tables 1.7 and 1.8). 
a) Use the data to graph T as a function of CO,. Graph in the same figure 
three linear functions. The first and second hnear functions have to 
provide an upper and lower bound for T as a function of CO2. The third 
linear function, which presents the best model, has to show the average 
of the lower and the upper bound. 
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t CO. 


T 


t CO2 


T 


t CO, 


T 


1961 317.64 


-0.044 


1977 333.78 


0.059 


1993 356.95 


0.102 


1963 318.99 


-0.047 


1979 336.78 


0.056 


1995 360.62 


0.276 


1965 320.04 


-0.219 


1981 340.11 


0.130 


1997 363.47 


0.355 


1967 322.16 


-0.151 


1983 342.84 


0.187 


1999 368.14 


0.262 


1969 324.62 


-0.004 


1985 345.87 


-0.018 


2001 371.07 


0.400 


1971 326.32 


-0.178 


1987 348.98 


0.167 


2003 375.78 


0.457 


1973 329.68 


0.060 


1989 352.90 


0.096 


2005 379.76 


0.479 


1975 331.08 


-0.134 


1991 355.48 


0.197 


2007 383.71 


0.404 


b) Present the three linear functions according to the formula 




^ CO.-L 


2 










T = = — 


-±c, 











where a, h, and c are constants that you have to provide. 

c) Discuss the advantages and disadvantages of this model. 

d) Consider T = 0.4°C and CO2 = 384 ppmv at t = 2007. Assume that the 
CO2 concentration increases by 10% in comparison to CO2 = 384 ppmv. 
What will be the corresponding temperature increase compared to T = 
0.4°C? You can neglect the temperature uncertainty. 

1.5.2 Consider the following data for the global temperature anomaly T and 
atmospheric CO2 concentration (see also Tables 1.7 and 1.8). 

t 1995 1997 1999 2001 2003 2005 2007 

CO2 360.62 363.47 368.14 371.07 375.78 379.76 383.71 

T 0.276 0.355 0.262 0.400 0.457 0.479 0.404 

a) Use the data to graph T and CO, as functions of t. Graph in the CO2 
figure a linear function that approximates the CO2 data. Graph in the T 
figure three linear functions. The first and second linear functions have 
to provide an upper and lower bound for T as a fiinction of t. The third 
linear function, which presents the best model, has to show the average 
of the lower and the upper bound. 

b) Present the linear functions according to the formulas 

C02= a +b{t -2007), T = ^— + c±d, 

e 

where a, h, c, d, and e are any constants that you have to provide. 

c) Combine COjCO and T{i) to a function T{COt) according to the formula 

,^CO^^_ 
B 

where A, B, and C are any constants that you have to provide. 
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d) Consider T = 0.4°C and COj = 384 ppmv at t = 2007. Assume that the 
CO2 concentration increases by 10% in comparison to COi = 384 ppmv. 
What will be the corresponding temperature increase compared to 7 = 
0.4°C? You can neglect the temperature uncertainty. 

1.5.3 Consider the same data as for problem 1.5.2. Use the CO2 and Tdata in the 
table to generate averaged CO^ and T data by averaging over three points. 
For example, the new 7(1997) = [7(1 995) + 7(1 997) + 7(1 999)]/ 3, the new 
7(1999) = [7(1997) + 7(1999) + 7(2001)] / 3, and so on. You will find five 
new data values for CO2 and 7 in this way. Use the new data values to do 
a), b), c), and d) of problem 1.5.2. 



2 Stochastic Analysis of Observations 



The basic steps of developing a mathematical model for any given observations 
(e.g., a model for the global atmospheric temperature increase) were described in 
Chap. 1. In particular, the goal was to find a relatively simple analytical function 
(for example a linear function possibly formulated in terms of redefined variables) 
that is capable of representing the trend of observations, and to adjust then the 
parameters of this function such that the model agrees as good as possible with the 
observed data. The adjustment of model parameters was performed in this way on 
a purely empirical basis. Consequently, there is the question of how it is possible 
to optimize the performance of models by finding model parameters that provide 
an optimal agreement between the model and observations. This question can be 
addressed, first, by introducing and minimizing a model error that accounts for 
differences between the model and data, or, second, by using assumptions about 
the sort of randomness involved in observations. The use of the second approach 
requires knowledge regarding the description and modeling of randomness. This 
approach will be described in Chap. 10 in conjunction with the introduction of 
probability concepts for several variables. The first approach for finding optimal 
model parameters will be described in this chapter. The discussions of probability 
concepts in the following chapters are prepared in this way by demonstrating the 
relevance and application of some basic tools for the description of random data 
like the mean, variance, and correlation coefficient. 

Section 2.1 explains the need for the development of optimal models. There is 
not only one optimization concept, but there are several concepts that can be used 
for the construction of optimal models. Essential advantages and disadvantages of 
different optimization concepts will be described in Sect. 2.2. The optimization of 
linear, quadratic, power and exponential function models will be addressed in the 
Sects. 2.3, 2.4, and 2.5, respectively. Section 2.6 deals with a summary of obser- 
vations made in this chapter regarding the design of optimal models. 
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2 Stochastic Analysis of Observations 
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Fig. 2.1. The increase of the atmospheric CO2 concentration in time V. (a) the CO2 
concentration in ppmv, and (b) the change S = [COj - CO-^t^\ lit- tg), where Iq = 
1959 and COjC'o) ^316 ppmv are used. The dots show measured data. The lines 
represent the models (1.28) and (1.29): see Chap. 1. 



2,1 Motivation 



Atmospheric CO2 Concentration. Let us consider again the development of 
the atmospheric CO2 concentration discussed in Sect. 1.5.2 to illustrate the need 
for the development of optimal models. The measured CO2 concentration and the 
change 5 = [CO2 - C02(?o)] / (^ ~ ^0) of the CO2 concentration in time t are shown 
in Fig. 2.1 together with the models derived in Chap. 1. We see that the models 
are certainly appropriate, but there is no guarantee that the models applied are the 
best possible models, this means optimal models. The latter question is relevant 
regarding the importance of the global warming problem. It is also worth noting 
that there are not only a few problems (like the CO2 concentration) that require the 
calculation of optimal model parameters. Instead, all the models developed on the 
basis of observations may benefit from an optimization. For example, Kepler's 
Third Law model derived in Sect. 1.2.2 agrees extremely well with observations, 
but there are also deviations between the model and observed data. Thus, the use 
of optimization techniques will be helpful for the improvement of the accuracy of 
the formulation of Kepler's Third Law (in particular for the calculation of the 
standard gravitational parameter). However, which concept can we use for opti- 
mizing the performance of models? One way to derive optimal model parameters 
would be the calculation of the relative error of several models and the use of the 
model that has a minimal relative error. However, the application of this approach 
is rather expensive: it requires the evaluation of a variety of models. There are also 
other approaches (see Sect. 2.2) that may be more appropriate. Thus, the question 
of the optimization of model parameters is a non-trivial question. 
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Questions Considered. The discussion in the preceding paragraph showed that 
there are several questions that have to be addressed: 

• What is the best definition of a model error that has to be minimized? 

• How difficult is the application of the best optimization concept? 

• What is the advantage of using a concept for finding optimal model parameters? 
The latter questions will be considered in the following sections of this chapter. 



2,2 Model Errors 

The minimizing of deviations between a model and observed data requires the 
definition of a global model error that becomes minimal. The term global means 
that we do not consider the local deviations at each data point, but we consider 
one error that represents a characteristic measure for all the local deviations. A 
closer look at this question shows that there are many ways to introduce a global 
error. Let us consider some characteristic advantages and disadvantages of several 
concepts for addressing this question. 



2.2.1 Model Errors 

Global Error Definitions. A first possibility for defining a global error is to 
follow the idea of finding the model for which the largest local relative error is 
minimal. However, instead of considering normalized errors we define the global 
error by the maximal absolute value of all local deviations 7, - yt,^X^ between 
given data 7, and the values of a model yj^x) at the positions X^ considered, 

E,=m3.x%-y^{X,)\. (2.1) 

Here, A' refers to the number of data points considered. This global error is called 
the Chebyshev error. A second possibility is given by using the mean value of the 
absolute values of 7, - y^iX^, 

E,=-i:%-yAXd[ (2.2) 

iV ,=1 

This error is called the least-absolute-deviations error E^. A third possibility is to 
consider the mean value of squared deviations Y^ - yJJC^, 

E'-\-hY,-yM(^X,)y. (2.3) 
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Fig. 2.2. The use of the Chebyshev criterion for 
the model optimization. The (Xj, Y^ data (2.4) 
are shown by dots for r = 0.2 and A^ = 10 in (c). 
The E(, and dE,^! db variations with b are shown 
for a = 0.8 in (a) and (b), respectively. The line 
in (c) shows the linear model (2.7). 



This error is called the least-squares error. The consideration of E^ refers to the 
fact that the mean value of squared deviations is obtained in this way. Obviously, 
there are many other possibilities to introduce global errors. For example, the 
second power in Eq. (2.3) can be replaced by any other power. 

Noise Model. Which global error definition will be the best choice for finding 
optimal model parameters? To address the latter question we will apply the three 
global error definitions referred to above to the modeling of the following set of 
data values {X-^, Y^, 



X, = i, 



(l -(-!)'>)/ 



(2.4) 



Here, i= \,N, where A^ refers to the number of values {Xj, 7,). For simplicity, we 
do only consider the case that A^ is an even number. Equation (2.4) represents a 
simple model for noisy data. The parameter r > characterizes the noise rate, for 
example r = 0.2 corresponds to 20% noise. The zero noise case is given by 
Xf = y, = /. y, values for odd / have positive deviations r / from the zero noise case, 
and Y^ values for even ; have negative deviations -r i from the zero noise case. The 
data are illustrated in Fig. 2.2 for the case that A'^= 10 and r = 0.2. The data applied 
suggest the use of a linear model function yj^x) for modeling the data, 

■ ax + b. 



yu 



(2.5) 



Let us consider now the properties of three error definitions that can be used for 
the calculation of optimal model parameters a and b in Eq. (2.5). 
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2.2.2 The Chebyshev Error 

Chebyshev Error. First, we consider the model that results from the use of the 
Chebyshev error definition for the optimization of the parameters a and b of the 
linear model Eq. (2.5). For the {X^, 7,) data (2.4) considered, the Chebyshev error 
is given by 

E^ = maxll^ "JmC^iOI = max (l - (-1)' r)i- ai-b\. (2.6) 

For given values of r and A^, we can find optimal linear model parameters a and b 
numerically by determining the a and b values for which E^, becomes minimal. 
By analyzing the dependence of the optimal a and b values on the parameters r 
and A' of the noise model (2.4), we find a= \ - r and b = r {N - \) m this way. 
Thus, the optimal linear model is given by 

y^={\-r)x + r{N-\). (2.7) 

The minimal value of ii^ for the optimal a and b values is given for A^ > 2 by 

E^.=r{N-l). (2.8) 

Ease of Use. Is it difficult to use the Chebyshev criterion for the optimization 
of model parameters? The variation of the Chebyshev error is^ with the linear 
model parameter b is shown in Fig. 2.2a, where the optimal value a= I - r = 0.8 
is used for the parameter a. This figure is only an example: the corresponding 
variation of E^ with the parameter a looks similar. In agreement with the conclu- 
sion about the optimal b value, E^. attains a minimum at^ = r(A^- 1)= 1.8. A 
disadvantage of using the Chebyshev criterion is that the optimal model param- 
eters a and b can only be found numerically. The reason for that is explained in 
terms of Fig. 2.2b, which shows that the corresponding derivative dE^I db ']Vin\ps 
?iib= 1.8. Thus, we cannot analytically determine optimal a and b values because 
we cannot find the values for which the derivative dE^I db = 0. 

Performance of the Criterion. The suitability of models obtained by using the 
Chebyshev criterion for the optimization is illustrated in Fig. 2.2c, which shows 
Eq. (2.7) for the case considered. This figure shows that the use of the Chebyshev 
criterion results in a curve such that 

^2 - yM iX,) = Y,-y^ {X,) = Y,-y^ (XJ = Y,-y^ {X,) 

The model (2.7) minimizes the maximum local error, but is does not pay attention 
to the data trend. In particular, the model behavior disagrees with the data trend 
for small / values. For A'^ — > oo, Eq. (2.7) implies that fo — > oo, which means that the 
data trend for small / values is not represented at all. Therefore, the use of the 
Chebyshev criterion is often found to be not the best choice. 
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Fig. 2.3. The use of the least-absolute -devia- 
tions criterion for the model optimization. The 
{Xj, y,) data (2.4) are shown by dots for r = 0.2 
and A^ = 10 in (c). In (a) and (b), variations of 
E^ and dE^i I db with b are shown for a = 0.76; 
(c) model (2.11) is shown by a line. 
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2.2.3 The Least-Absolute-Deviations Error 

Least-Absolute-Deviations Error. Next, let us have a look at the suitability of 
the least-absolute-deviations error E^ for the optimization of the model (2.5). For 
the {Xj, y,) data (2.4) considered, E^ is given by 






(2.10) 



The optimal model parameters a and b that minimize £4 for given r and A^ values 
can be found by analyzing the dependence of E^ on a and b. By considering the 
effect of r and A'^ variations one finds the expressions fl = l-r(A'+l)/ (A^- 1) and 
b = 2Nr(N- 1). Hence, the optimal linear model is given by 



yM 



, N + \ ] 2N 

1 r \x + r . 

A'-l j N-\ 



(2.11) 



The minimal value of ii ^ for the optimal a and b values is given for A'^ > 2 by 



E,=r 



N 



N + \ 
N-V 



(2.12) 
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Ease of Use. The difficulty of applying the least-absolute-deviations criterion 
for the optimization of model parameters is addressed in terms of Fig. 2.3. The 
variation of E^ with the linear model parameter b is shown in Fig. 2.3a, where the 
optimal value a = 1 -r{N+ 1)/{N- 1) = 0.76 is used for the parameter a. E^ attains 
a minimum at b ^ 2Nr(N- 1) = 0.44. The curve E^ looks like a smooth function, 
but Fig. 2.3b shows that this view is incorrect: the derivative dE^ I db is not a 
smooth function. Consequently, the optimal a and b values cannot be calculated 
analytically. This fact is definitely a significant disadvantage of the least-absolute- 
deviations criterion. 

Performance of the Criterion. The suitability of models obtained by using the 
least-absolute-deviations criterion for the optimization is illustrated in terms of 
Fig. 2.3c, which shows the curve (2.11) in comparison with the data considered. 
This curve (or a closer look at the consequences of Eq. (2.11)) shows that the 
least-absolute-deviations criterion provides a curve where 

yAX,) = Y„ y^{X,) = Y,. (2.13) 

For the case considered it turns out that the use of the least-absolute-deviations 
criterion ends up in a result that can be obtained without using any optimization: 
any two points can be chosen to define the linear model. There is no reason to 
assume that the use of the data values at / = 1 and / = A' is better than other 
choices. Obviously, such a concept is not helpful in general, which means that the 
use of the least-absolute-deviations criterion is often found to be not the most 
convenient choice. 



2.2.4 The Least-Squares Error 

Least-Squares Error. Next, let us consider the properties of the least-squares 
error is. For the {Xj, 7,) data (2.4) considered, E^ reads 

E'-\-t{y^-yMiX,)f^\-t[{i-{-\yr)i-ai-b\\ (2.14) 

The calculation of the optimal model parameters a and b that minimize the error 
E^ is shown in the beginning of Sect. 2.3.3. According to Eq. (2.61) we obtain 
a = 1 - 3 r / (A'- 1) and b ^ r{N+2) I {N-\). Consequently, the optimal model is 
given by 

Ti 3r ^ iV + 2 

"^ I N-\) N-\ 
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Fig. 2.4. The use of the least-squares criterion 
for the model optimization. The {X^, 7,) data 
(2.4) are shown by dots for r = 0.2 and A' = 10 
in (c). (a) and (b) show the variations of ii and 
dE I db with Z) for a = 0. 93 . The line in (c) shows 
the linear model (2.15). 



The minimal value of is^ for the optimal a and b values is given for # > 2 by 



2 (A''-4)r' 2A' + 1 



E' ^ 



N-l 



(2.16) 



Ease of Use. Figure 2.4 illustrates the suitability of using the least-squares 
criterion. The variation of E with the linear model parameter b is shown in 
Fig. 2.4a, where the optimal model parameter a = 1 - 3 r I {N - 1) = 0.93 is used. 
E attains a minimum at the optimal value b = r {N+ 2) / (A'- 1) = 0.27. Figure 2.4b 
shows that the derivative dE/ db is a smooth function. Therefore, optimal a and b 
values can be found analytically (see Sect. 2.3). The latter feature of the least- 
squares criterion represents a significant advantage of this concept. 

Performance of the Criterion. Figure 2.4c illustrates the suitability of models 
obtained by using the least-squares criterion for the optimization. The curve 
agrees exactly with the first data point. 



yMi^^) = Yr 



(2.17) 



The model (2.15) does not reveal the disadvantages of the Chebyshev and least- 
absolute-deviations criteria: it represents the data trend very well, and the model is 
not simply a curve through any two data points. Therefore, both the good model 
performance and the property to provide analytical expressions for optimal model 
parameters suggest the use of the least-squares criterion for the optimization of 
models. 
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2.3 Optimal Linear Models 

Due to shortcomings of alternative concepts that were explained in Sect. 2.2, 
only the least-squares error concept will be used in the following to determine 
optimal parameters of models. In this section, we will show how the least-squares 
error concept can be used for finding optimal linear functions. Before addressing 
the latter question we introduce in Sect. 2.3. 1 useful notation for the description of 
properties of random data. The use of such definitions of means, variances, and 
correlations enables an efficient representation of optimal model parameters. We 
will use these definitions as abbreviations in this chapter. Detailed explanations of 
the properties of one and several random variables are provided in Chaps. 4 and 
10, respectively. 



2.3.1 Means, Variances, and Correlations 

Mean Values and Fluctuations. We consider data values {Xj, 7,), where 
i= \,N, this means we have A' observations of any two variables X and Y. An 
example for such a data set is given by the data of the noise model (2.4), which are 
shown in Fig. 2.4. A basic characterization of the variables X and 7 is given by the 
mean values (or expectation values), which are defined by the relations 

■\ N 1 W 

{X) = —TX-, (y) = —YY-. (2.18) 

In general, X^ and 7, are unequal to their means <X> and <Y>, respectively, but 
there are deviations from the mean values. Such deviations from means (which are 
called fluctuations) are defined by 

X,=X^-{X), Y, =¥,-{¥). (2.19) 

According to their definition, the mean values of fluctuations disappear, 

^> = ^Zk - (^)) = (^) - (^) = 0, (2.20a) 

Y) = ^t{Y,-{Y))={Y)-{Y) = 0. (2.20b) 

Variances. To understand the relevance of fluctuations we may consider mean 
values of quadratic variables, which are called second-order moments, 

X-) = —fX-X-, (Y-) = —fY.Y-, (XY) = —fx.Y. (2.21) 
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The problem related to the consideration of the latter variables is that they do not 
directly provide the information which we are interested in: information about the 
amount of fluctuations involved. A more appropriate measure for the intensity of 
fluctuations is given by the mean values of quadratic deviations from the mean, 



XY 



1 ^ 



-{{x-{X)){Y-{Y))) = j^Zy^-{X)){Y,-{Y)), 



(l^) = ((x-(X))^) = i^g(x,.-(X))(x,-(X)), 
Y') = {{Y-{Y)Y) = lt{Y.-{Y)){Y.-{Y)). 



(2.22a) 
(2.22b) 
(2.22c) 



These variables can be written as functions of the moments defined by Eq. (2.21), 



X'] = (X'-2X(X) + (XY 



Y^ -2Y(Y) + (Y) 



Xy\ = (XY- X{Y) - Y{X) + {X){Y)) = [XY) - {X){Y) 



X^)-{X) , (2.23a) 

Y-)-(Y)\ (2.23b) 

(2.23c) 

These relations do explicitly demonstrate that the second-order moments involve 
information about both the amount of fluctuations (given on the left-hand sides), 
and mean values (given by the last terms). The variables on the left-hand sides are 
called variances. The variances represent squared variables, i.e., the variances do 
not directly characterize the range of variations of the random data X and Y. The 
latter information is provided by the standard deviations <X^ >"" and < y ^ >"^ of 
Xand Y, respecfively. 

Noise Model Example. An illustration for means and variances defined in this 
way is given by the expressions that follow from the noise model (2.4), 
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(2.24) 



These expressions are found by making use of Eqs. (2.18) and (2.22) for the cal- 
culation of these quantities. The means <X> and <Y> and the ranges <X> ± 
<jf2>i/2 ^j^j ^Y> ± <y2>i/2 of variations of data values indicated by the corre- 
sponding standard deviations are shown in Fig. 2.5. The range of variations 2 
<X^>"^ and 2 <7^>"^ considered in this way does not cover all the data 
variations. Instead, the standard deviations do only provide the order of magnitude 
of data variations (see the more detailed explanations in Chap. 4). 
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Fig. 2.5. An illustration of means and standard 
deviations of the noise model. The {X^, Y,) data 
(2.4) are shown by dots for r = 0.2 and A' = 10. 
The vertical lines show <X> (solid line) and 
<X> ± <x2>i/2 (^^ashed lines). The horizontal 
lines show <Y> {solid line) and <Y> + <Y^ >"^ 
{dashed lines). 
2 4 6 8 10 12 

Correlation Coefficient. Regarding the consideration of two random numbers 
X and Y, it is relevant to know in which way both variables are related to each 
other. This information is covered by the cross variance <XY >. The most appro- 
priate way to look at < XF > is to consider the nondimensional variable 
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(2.25) 



which is called the correlation coefficient. This coefficient has the property to be 
bounded by -1 and 1, 

-l<r^,<l. (2.26) 

To show the validity of this relation we introduce a non-negative function H{p) > 
by the definition 



H{p) = ({x + pYf\ = (x') + 2p[xY) + p'lY'). 



The first two derivatives of H{p) by the parameter j9 are given by 



f = 2(xy).2p(y'), 



d^H 
dp 



2Y'). 



(2.27) 



(2.28) 



These two derivatives show that 77 has a minimum value atp^^ -<XY >/ <Y^> 
for which the first-order derivative is equal to zero (provided < 7 ^ > > as will be 
assumed here). The minimum //,^i„ of H{p) is given atp^ by 



//,. 



X^)- XY 



(2.29) 



The function H{p) > for all p. Thus, H{p^) = /4,i„ also has to be non-negative, 
//,^i„ > 0. Consequently, we find 



XY 



X')Y' 



<l 



(2.30) 
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Fig. 2.6. The noise model: the dependence of the 
correlation coefficient r^ on the noise rate r. The 
solid line shows the asymptotic curve (2.34). The 
dashed lines show the correlation coefficient in 
dependence on the sample number N according to 
Eq. (2.33). Short dashes correspond to A^= 100, 
long dashes to N^ 10. 



which implies the relation (2.26) if the definition (2.25) of I'xi- is accounted for. 
The inequality (2.26) represents Schwarz's inequality. Under which conditions do 
we find specific values of the correlation coefficient? A first specific case is given 
if y, is independent of Jf|, this means 7, is not a function of J^ (such variables are 
called independent variables, see Chap. 4). In this case, r^Y is given by 



("> 



X)Y 



X')Y 



X')Y 



(2.31) 



A second specific case is given if 7, is a linear function oiXj, Yj = Ci J^ + Cj, such 
that y, is fully defined by X^. Here, the coefficients c, and c, are any constants. In 
this case, the mean of 7, is given by <Y> = Ci <X> + Cj, and the fluctuation is 
Y- = c^Xj ■ Thus, the correlation coefficient is given by rxy ^ cj |C||, which means 



±1 



(2.32) 



depending on the sign of c^ It is worth noting that \r)^\ ^ 1 if 7, is a function ofX^ 
but not a hnear function (see the discussion of Fig. 2.7). Thus, the value of r^y 
indicates the degree of linear correlation of two random variables. 

Noise Model Example. Let us consider again the noise model (2.4) to illustrate 
the variations of rxy- By using Eq. (2.24) we find for the correlation coefficient the 
expression 
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(2.33) 



For the case that A' is sufficiently large, r^y approaches the limit 
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(2.34) 
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Fig. 2.7. Correlation coefficients for problems 
discussed in Chap. 1 . (a), (b), (c), and (d) show 
data of CO2 concentrations, Kepler's Third Law, 
the U.S. energy consumption, and the global 
temperature anomaly; (e) temperature data cal- 
culated by Eq. (2.35). 
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The functions (2.33) and (2.34) are illustrated in Fig. 2.6. The curves shown in this 
figure support the conclusions of the discussion of the limit cases I'xy = ±1 and 
Fx^- = in the preceding paragraph. For r = (i.e., for 7, = X^, we have r^ = 1 . For 
r — > 00, Yj becomes independent of X,. Hence, the correlation coefficient r^^Y ~^ in 
this case. It is interesting to observe that for the sample numbers A'^ considered the 
correlation coefficient is close to one for relatively small r^y. For example, we find 
relatively high values r^ < 0.98 for r < 0. 1, which means for a noise of 10%. The 
correlation coefficient increases with the number of samples as long as the asymp- 
totic curve (2.34) is not reached. 

Other Examples. The correlation coefficients of data considered in Chap. 1 are 
shown in Fig. 2.7. It was shown in Chap. 1 that the COj concentration data and the 
period data related to Kepler's Third Law can be modeled very accurately (the 
relative error of the corresponding models was below 1%). Thus, these data are 
hardly affected by randomness, i.e., the minor deviations of these correlation 
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coefficients from i'xy ^ 1 arise, first of all, from the deviations of these data from a 
linear trend. Regarding the U.S. energy consumption, the deviation of the corre- 
lation coefficient from one is implied by the data randomness. It is worth noting 
that the correlation coefficients in Figs. 2.7a-c are relatively high. Figure 2.7d 
shows the global temperature anomaly data that are significantly affected by both 
nonlinearity and randomness. The relevance of nonlinearity and randomness is 
addressed in terms of Fig. 2.7e. This figure shows the same number of data as Fig. 
2.7d, but the temperatures are now calculated from Eq. (1.33), 



r = -0.35 + 



^-1840 
178 



(2.35) 



The correlation coefficient of these non-random temperature data is r^y ^ 0.88. 
Hence, the main reason for the deviation of the correlation coefficient from one in 
Fig. 2.7d is the nonlinearity. These examples support the conclusion of the noise 
model shown in Fig. 2.6: it requires a very high degree of randomness to observe a 
very small correlation coefficient close to zero. 

Skewness and Flatness. Means, variances, and the correlation coefficient are 
the quantities involved in the optimization of linear functions. The optimization of 
quadratic functions in Sect. 2.4 leads to the need to consider other quantities in 
addition, like the skewness m^ and the flatness m^ that are defined by 



X') X' 
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X') X 



(2.36) 



To prepare the use of m^ and m^ in conjunction with the optimization of quadratic 
functions we derive here a relevant relation between these quantities on the basis 
of the property r^^ < 1 of r^y- By replacing in the definition (2.25) of correlation 
coefficients X by XV < X" > -1 and 7 by X/ < X^ >"' , we find 
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The use of the definitions of m^ and m^ then results in the inequality 

m^ +1<»74. 




(2.38) 



Under which condition does the equality sign appear here? In correspondence to 
the conditions for rxi' = 1, we find this case if X/ < X^ >"^ is a linear function of 
X^ / <X^ >-l, this means 
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(2.39) 
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The requirement to use the proportionality factor 1 / m^ can be seen by multiplying 
Eq. (2.39) with X I < X^ >"^ and taking the average of both sides. The claim that 
this relation implies the equality sign in Eq. (2.38) can be proven by the multipli- 
cation of Eq. (2.39) with X' I <X^' > and averaging: we obtain m^ = {m^ - 1) / m^ 
in this way. A better understanding of the requirement related to Eq. (2.39) can be 
achieved by multiplying it with <X^ >^ . We find 



X'')X = {X^)\X^ -{X-)i (2.40) 



where the definition of m^ is applied. This relation represents a quadratic equation 
for X . By solving this equation, one finds a nonfluctuating value for X . The only 
possibility that X does not fluctuate is J^ = 0, which means the equality sign in 
Eq. (2.38) can only appear in the absence ofXj fluctuations. Hence, we have 

0<m^-\-m^\ (2.41) 

if < Jf ^ > > 0, as will be assumed here. 



2.3.2 Optimal Linear Functions 

Error Formula. Let us assume that there are i = \, A'^data points {X^, 7,) that 
follow, basically, a linear function (for example, the data points given in Fig. 2.4). 
Our goal is to determine the parameter a and b of the linear model 

y^=ax + b (2.42) 

such that this linear fiinction does optimally agree with the data. In particular, we 
look for a and b parameter values that minimize the least-squares error 

^' -T7Z(^ -yMiX,)f=-Y^{Y,-aX, -bf. (2.43) 

l\ ,=1 i\ ,=1 

The error E^ can be rewritten by distributing the sum, 

] N ] N I'V ] N ] N 

E^=—TY--2a — TXY.+a-—TX^-2b — YY.+2ab — TX-+b^ 

Nik ' nU • • Ntt ' Ntt ' Ntt ' 

= (Y")-2a(XY) + aHx^)-2b(Y) + 2ab(x) + b^ 



[Y') + {Yy-2a({xY) + {x){Y))+a'({x') + {xy)-2b{Y) + 2ab{x) + b' 

= (?')- 2a(^XY^ + a^lx^'^ + b^ - 2b(Y) + [Y^ + 2a{x){b - {y))+ a^{x)' 

= (Y-\-2a(xY\ + a'~!x^\ + {b-{Y) + a{X)y. 

(2.44) 
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Here, the second line makes use of the definitions of means (2.18) and second- 
order moments (2.21). The third line applies Eqs. (2.23), which relate the second- 
order moments and variances. The fourth line reorganizes the ordering of these 
terms to prepare the final result that combines the last five terms of the previous 
line in one quadratic term. 

Critical Points. The mean values involved here are any numbers that can be 
calculated from the data points {Xj, 7,). Thus, the error E^ is a function of the 
model parameters a and b. A required condition to find a minimum or maximum 
of E' is given by the need that the partial derivatives of E^ by a and b disappear 
(the partial derivative ofE^ by a is given by keeping b constant and differentiating 
E^ by the parameter a). These derivatives are given by 



BE 



2(^XY^ + lalx^'j + 2{x){b - (y) + a{x)), (2.45a) 

^ = 2{b-{Y) + a{x)). (2.45b) 

The requirement that these partial derivatives have to disappear then results in the 
following critical points (the last term in (2.45a) disappears because of (2.45b)) 

(xy) 

« = W, (2.46a) 

b = {Y)- a{x). (2.46b) 

The linear model that results from the use of these a and b values is given by 

(xy) 



>'M=(^) + Wy(--W)- (2-47) 

Here, the model parameters can be calculated from given data. 

Minimum. At this point, it is still unclear whether E^ has a minimum for these 
a and b values, i.e., whether Eq. (2.47) represents an optimal model. This question 
can be addressed by means of the Second Derivatives Test of Calculus. We define 
a variable D by the relation 



D 



d\E^d\E^ 
da^ db^ 



da db 



(2.48) 



According to the Second Derivatives Test, E^ has a minimum at the critical points 
(2.46)ofaandZ7if 

-^^>0, and D>0 (2.49) 

da' 
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at the critical points. According to Eq. (2.45), the second-order partial derivatives 
involved in the definition of Z) are given by 

''''. = 2lx^.2lX)\ ^ = 2, ^.^.2(X). (2.50) 



da' \ I ' ' dh' dadh dbda 

Hence, d^E' I da' > for all values of the parameter a. The use of Eqs. (2.50) in the 
definition of Z) shows that 

Z) = 4/1'\ + 4(X)'-4(X)' =4(X'\, (2.51) 

which means that £) > under the condition that < X* > > 0, as it will be assumed 
here. This means that E' has indeed a local minimum at the critical values (2.46) 
for a and h, which means that Eq. (2.47) represents the optimal model. 

Minimal Error. The minimal value of the least-squares error E' can be found 
by using the critical values for a and h in Eq. (2.44), 

(ir) ,_, Ixy)^ ,^ , (xy) 
x'V I x'\ \ / X' 



E'=(P)-2\—{-{XY) + \^ = {P)-\^. (2.52) 



E' is completely determined by the data properties. Hence, E' does not say any- 
thing about the standard of the linear model applied, which means that E' is not 
helpful for the evaluation of the suitability of models. Therefore, no attempt is 
made to use E' regarding the applications considered in the following. An evalu- 
ation of models obtained can be performed by considering the relative model 
error, as done in Chap. 1. Rewriting of Eq. (2.52) for the error E' provides 

E'=(Y')(l-r„'), (2.53) 



where the definition (2.25) of the correlation coefficient r^y is used. This relation 
shows that E' is proportional to <Y'>, which is a measure for the typical amount 
of the Y data randomness. E' is also proportional to 1 - r^*. According to the 
discussion of properties of the correlation coefficient in Sect. 2.3.1, the limit case 
1 - rxY^ = is given if 7, is a linear function of J^, and the limit case 1 - r^-^ = 1 is 
given if Y^ is independent of X^. Hence, the stronger the dependence of 7, and J^, 
the smaller is the error E'. 

Modification. The approach presented here to derive the optimal linear model 
(2.47) can be also applied to other linear optimization problems. For example, we 
may have the problem that there is a relatively complicated function, and we are 
interested to approximate this function over a certain interval by a linear function. 
A problem of this kind was considered in Sect. 1.5.3, where we tried to find a 
linear approximation to the nonlinear temperature-C02 relation (1.38). Instead of 
addressing this problem empirically by looking for an appropriate reference point 
for the Taylor expansion, we may apply the idea of linear optimization described 
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above to solve such a problem in a much more convenient way. Let F{x) refer to a 
given complicated function that we would like to approximate by a linear function 
paiN) = i-ij( + iy^ Hence, we look for the parameters a and b such that the relation 

F{x) = ax + b (2.54) 

is optimally satisfied. A first condition for a and b arises from the constraint that 
Eq. (2.54) has to be satisfied in the integral mean. The integral mean over an 
interval between c and d is defined for any variable Q (which can be F or x) by 

{Q)^=-L-]Q(x)dx. (2.55) 

The subscript / of the symbol <Q>i refers to the integral mean used here instead of 
the mean (2.18) used above. By integrating Eq. (2.54) between c and d and 
dividing both sides by ^ - c we find 

{F)^=a{x)^+b, (2.56) 

where we made use of the fact that <1>/ = 1. We take the difference between the 
Eqs. (2.54) and (2.56) to have an equation that does only involve a, 

Fix)-{F)^=a[x-{x)^). (2.57) 

The direct integration of this relation does not make sense because we end up with 
= 0. Thus, we multiply this condition with x - <x>, and integrate after that. This 
leads to the following condition for the parameter a, 



(-WJ 



n-i^)' 



The comparison of the optimal parameters a and b obtained by Eqs. (2.58) and 
(2.56) with the optimal parameters given by Eq. (2.46) reveals the correspondence 
of the results. The integrals over x involved here can be calculated - we obtain 

/ V I i , \ d^ -c^ d + c 

-\xdx- 



^ '' d-d 2 d-c 2 



(-WJ 



2\ / 2\ ,y_ld^-c' (d + cf _l{d-cy+3d^c-3dc^ 



I \ /' ^ " 3 d-c 4 3 d- 



c 



id.cr_id-cr^^^_iji-c)i_^^jji-ci ^23^^ 



4 3 4 12 

By using these expressions we obtain for a and b inF^^"^' = ax + b the relations 



12 



{d-cf 



{^'>,-n^ 



/,.(/'), -»^. (2.60) 
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2.3.3 Examples 

Let us illustrate the use of optimal linear models. In particular, we will consider 
three examples. First, we consider the modeling of the noise model data (2.4), for 
which the optimal linear model can be directly applied. Second, we will consider 
Kepler's Third Law data, which requires a data transformation. Third, we will 
illustrate the application of the optimal linear model (2.54) to continuous func- 
tions by the derivation of a linear temperature-C02 relation (see Sect. 1.5.3). 

Noise Model. The construction of an optimal linear model for the noise data 
(2.4) requires the use of Eqs. (2.24) for means and variances in Eqs. (2.46) for a 
andfo. 



a = ' 



XY) 12 N'-l 



X-) N-\ 12 



3r 



b = (Y)-a(X] ^ + 1 ' ^ + 1 



2 2 
r2N+4 N+2 



= 1- 


3) 


N- 


3/ 


- 


N- 


-1_ 



A^-l N-V 

r 3{N + 1) - (A^ - 1) 



2 A^-l 



(2.61a) 



(2.61b) 



2 A^-1 N-\ 

Therefore, the optimal linear model is given for this case by 

r, 3r ^ N + 2 
y,^=\l U- + r . (2.62) 

"^ \ N-\) N-\ 

Kepler's Third Law. Next, let us use the linear model (2.47) for the derivation 
of Kepler's Third Law discussed in Sect. 1.2.2. The data values are given in 
Table 1.2. According to Kepler's Law, the period Tp is related to the mean 
distance r from the Sun by a power function law, 

Tp^c/. (2.63) 

Here, c and d are model parameters that have to be determined. To enable the use 
of methods for linear equations, we write Eq. (2.63) as a linear function of the 
variables In Tp and Inr, 

InTp^dlnr + lnc. (2.64) 

Next, we introduce new variables to enable the direct use of the formulas derived 
in Sect. 2.3.2, this means write Eq. (2.64) as 

yj^=ax + h, (2.65) 

where the variables are defined by the relations 

y^f =lnTp, x = lnr, a = d, b = \nc. (2.66) 
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The optimal model parameter a and b can be calculated on the basis of Eq. (2.46) 
and the data from Table 1.2: the Inr data are the Xj data, and the In Tp data are the 
y, data. This calculation provides the optimal values a = 1.4995 and b = 2.8483. 
We use Eqs. (2.66) to find for c and d the optimal values 

c = 17.2584, d=\A995. (2.67) 

The optimal model for the period Tp is then given by 

7> =17.2584r''"". (2.68) 

To compare this result with Eq. (1.8) we use d= 1.5 and write 



7'p=J^'-^ (2.69) 

where G^ = 0.1325 (10' km)^ / a^. This result is basically the same as the result 
G^= 0.1324 (10' km)Vfl' derived in Sect. 1.2.2. Thus, the optimal model (2.69) 
confirms the result derived in Chap. 1. 

Linear T-CO2 Relation. The third example considered is the derivation of a 
linear temperature-COj relation by means of the optimal linear model for continu- 
ous functions. Equation (2.54) reads for this case 

T = aCO^+b. (2.70) 

Here, the temperature Tis determined by the function (1.38), 



r = -0.35 + 



119 47 
178 356 



^ ' 40 ^"'** 

1 + (CO, -316)-1 

329 



J 



(2.71) 



The application of Eqs. (2.60) for a and b requires the calculation of the integrals 
<F>i and <Fx>i over the interval considered (i.e., from c = 316 to d ^ 390). The 
convenient way for doing this is to perform the integration numerically. The latter 
results in 

(7^)^=0.1637, (7? x)^ =61.6325. (2.72) 

The use of Eq. (2.60) for the calculation of the parameters a and b in the linear 
approximation T"'^' = a COi + b for the temperature T leads then to the result 

j(UN) _ CO2 -333.5 
119.2 

This result is very close to the finding "P"^' = (COj - 333.3) / 120.6, which was 
empirically obtained in Sect. 1.5.3. Figure 2.8 shows an almost perfect agreement 
between the curves (2.71) and (2.73). The advantage of the approach presented 
here is that the optimal model parameters are determined for a given interval 
considered, i.e., there is no need for the adjustment of model parameters. 
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Fig. 2.8. The temperature-C02 relation. The solid 
line shows the exact relation (2.71). The dashed 
line shows the linear approximation (2.73). 



2.4 Optimal Quadratic Models 

Linear models have a limited range of applicability: there are many processes 
that cannot be described by linear functions. Correspondingly, the extension of the 
linear methods presented in Sect. 2.3 will be discussed next by addressing the 
question of how optimal quadratic functions can be constructed. The theory of 
optimal quadratic fiinctions will be described in Sect. 2.4.1. Two applications will 
be considered then in Sects. 2.4.2 and 2.4.3. 



2.4.1 Optimal Quadratic Functions 



Error Formula. The development of an optimal quadratic model is more chal- 
lenging than the development of an optimal linear model. To avoid complicated 
rewritings, it is the best to consider the following quadratic function, 



yM={Y 



A 



{-{x)Y 



X' 



B- 



X- X) 



C 



X' 



(2.74) 



This function can be rewritten into the standard form of a quadratic function, 

y,^=ax +bx + c, (2.75) 

where the model parameters a, h, and c are determined by the relations 



a X-) = a(y 



b X')= Y' 



B{X' 



-2A(X\ 



c{X' 



A(XY-B(X)(X' 



+ C X^)\ 



(2.76) 
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A, B, and C are unknown parameters that have to be determined. The means are 
defined by Eq. (2.18) and the variances by Eq. (2.22). The variances <Y^> and 
< X^ > are assumed to be unequal to zero, which means that there are Y^ and X^ 
fluctuations. The ratios (x - <X>) / < X^ >"^ and j'^/<y^>"^ are nondimensional. 
Hence, the model parameters A, B, and C are nondimensional, too. The advantage 
of writing Eq. (2.74) in this way is that the calculation of the model parameters A, 

B, and C becomes relatively simple. The least-squares error E^ is given for the 
quadratic function (2.74) by the expression 

A r nx2 



1 ^ 

— z 



A ^ ', J J' +B ' \,y +c 



X' 



Y ) N 



x; 



X' 



X- 



B- 



X' 



X, 



(2.77) 



X' 



C + A- 



Here, we used 7, = <5',> + ^ , and we wrote the second parenthesis term so that 
this term becomes proportional to X^ - <X^>, which vanishes in the mean. 
By distributing the terms of the sum, the normalized error can be written 
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x*-2xHx' 



-2B- 
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+ A' 



X^-iX^ 
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X' 
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C + A- 



(2.78) 



This formula can be written more efficiently as 

-1-2 Ar^ + A^{m^-\)-2Br^ +2 ABm^+ B^ 



Y- 



where we apphed the abbreviations 



C + A- 



X^Y 



X' 



,,1/2 ,~ >l/2 ' 'XXY 

X^\ Y^^ 



x^ y 



1/2 ' '"3 



X' 



■JY'"^4 



X' 



X' 



(2.79) 



-.(2.80) 



The correlation coefficient r^-, skewness m^, and the flatness m^ were already used 
in Sect. 2.3.1. The coefficient rj^^yy is defined in correspondence to r^y. 
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Optimal C. The optimal C value has to be chosen such that the non-negative 
bracket term in Eq. (2.79) disappears. Hence, the optimal C value is given by 



C 



■A. 



(2.81) 



By using this value we find the error to be given by 



-■ (B + Anij ~'"at)^ -(Am^ ~'';n')^ +^~^^''xxy + A^{m^ -l) 

■■{B + Am^-r„f +{m^-\-m^^)A- -2(r^^ -m^r^,)A + \-rx 



■-(B + Am^ -r„) + (m^ -I- nij ) 



'3' XY. 



m^-l-m^' J 



(2.82) 



+ l-rv. 
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■m^r^) 



win -I- m^ 



Optimal A and B. Optimal A and B values can be found in correspondence to 
the calculation of the optimal C value. Equation (2.41) shows that m^- I - m^ > 
if < X^ > > 0, as is assumed here. Therefore, the first two terms on the right-hand 
side of Eq. (2.82) are both non-negative. The optimal A and B are the values for 
which the first two terms are equal to zero, this means the optimal A and B are 



A = - 



-m.ry 



1^-l-m^ 



B = r^y - ffjj A. 



The minimal error that is given for these values reads 



1-r, 



('-;. 



■m,r^.) 



'J4 - 1 - m{ J 



(2.83) 



(2.84) 



The comparison with the error (2.53) of the optimal linear model shows that the 
error of the quadratic model is smaller than the error of the linear model because 
of the negative last term in Eq. (2.84). 



2.4.2 Vehicular Stopping Distance 



Optimal Function. As a first application of an optimal quadratic function we 
will consider the vehicular stopping distance problem discussed in Chap. 1. The 
data values are given in Table 1.3. According to the conclusions of Sect. 1.2.3, 
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Fig. 2.9. The stopping distance D (in feet) as a 
function of the car's velocity v (in mph). The 
5'o//fi? line shows the optimal quadratic model 
(2.86), and the dashed line shows the model 
(2.87), which was derived in Sect. 1.2.3. The 
data given in Table 1.3 are shown by dots. 
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it is reasonable to assume that the data can be described by a quadratic function 

D = av^ +bv + c. (2.85) 

The model parameter can be obtained by combining Eqs. (2.76) with the expres- 
sions for^, B, and C derived above. This calculation leads to the result 

(2.86) 



Z) = 0.0478i;'+ 2.1771;; + 0.4681 = 0.4681+ 2.1771+ \v. 

20.9107, 



The last expression represents a rewriting in correspondence to Eq. (1.9) obtained 
in Sect. 1.2.3, 



£>: 



20 



(2.87) 



Application. Figure 2.9 shows a comparison between the model (2.86) and the 
model (2.87) derived in Sect. 1.2.3. This figure does not show any observable 
difference between both model functions. Nevertheless, a difference is given by 
the constant 0.4681 that appears in relation (2.86). The latter means that we have 
at u = the stopping distance Z)(u =0) = 0.4681, which does not make sense. The 
way to construct an optimal quadratic model cannot account for such conditions. 
Such a problem can be avoided by developing an optimal linear model for £> / u in 
correspondence to the development in Sect. 1.2.3. 



2.4.3 COi Concentrations 

Optimal Function. As a second application of an optimal quadratic model we 
consider the modeling of the atmospheric CO2 concentration development, which 
was discussed in Chap. 1. The measured data are given in Table 1.8. As shown in 
Sect. 1.5.2, the increase of atmospheric CO2 concentrations can be described by a 
quadratic model 



CO, =at +ht + c. 



(2.88) 
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Fig. 2.10. Atmospheric COt concentration data. 
The Table 1.8 data are shown as dots. The solid 
line shows the optimal quadratic model (2.89); 
the dashed line shows the model (2.90), which 
was derived in Sect. 1.5.2. 



The model parameters can be determined as explained in Sect. 2.4.1. The result of 
this calculation is given by the function 



CO2 =0.0120r -46.1486? + 44690.4788 

( ?-1959^ 

= 3 15.2335 + 0.8446 1 + ^^ \{t - 1959). 

V 70.4158/ 



(2.89) 



The last line represents a rewriting of the first line in correspondence to Eq. (1.29) 
obtained in Sect. 1.5.2, 



CO2 =316 + 0.71 1 + 



'-''''^ii-X959). 
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(2.90) 



Application. Figure 2.10 presents a comparison of the optimal model (2.89) 
with the empirical model (2.90), which was derived in Sect. 1.5.2. The perform- 
ance of both models is very similar. It seems that the use of the empirical model 
(2.90) is more appropriate due to the very good agreement with the data values 
close \.o t= 1959. However, a closer look shows that the optimal quadratic model 
(2.89) provides a better overall prediction, in particular for data between t = 1980 
and t = 1990. Thus, the optimal model (2.89) actually is the slightly better model. 



2,5 Optimal Power and Exponential Models 



Linear and quadratic functions are applicable to many modeling problems, but 
there is also a variety of problems that cannot be handled on this basis. Therefore, 
we consider here the construction of optimal power and exponential functions. In 
some cases it is possible to rewrite such functions such that optimization methods 
for linear functions can be applied (see, e.g., the optimization of redefined model 
variables in Kepler's Third Law discussed in Sect. 2.3.3). This approach may 
become inapplicable if we have the (relatively usual) case that we have to account 
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for a reference point (xq, y^) as given in the functions 

>'m = Vo+a(x-x„)*. (2.91a) 

;.^=Jo+a(e"<"°'-l). (2.91b) 

Here, a and b are any model parameters. It may be seen that y,^ = y^ at x ^ Xq. It 
will be assumed that Xq is any known reference point - usually it is not difficult to 
find an appropriate value. For a known y^ value, we can write the power function 
model (2.91a) as a linear function of redefined variables, but this approach does 
not work regarding the exponential function model (2.91b). The modeling of the 
noisy data of the global temperature anomaly discussed in Sect. 1.5.2 showed that 
it may be impossible to find an appropriate value for yg for the power function 
model (2.91a), such that this model cannot be linearized. To handle such cases we 
need an approach that allows the direct optimization of Eqs. (2.91). The way to 
handle this question will be described in the following, this means we will find 
optimal Jo, a, and b parameter values for Eqs. (2.91). The problems considered 
here represent examples for the way to address such questions: there are many 
similar problems that can be solved correspondingly. The optimization of Eqs. 
(2.91) will be explained and illustrated in Sects. 2.5.1 and 2.5.2. An application to 
real data, the modehng of the global temperature increase, will be discussed in 
Sect. 2.5.3. 



2.5.1 Optimal Power Functions 

Error Definition. We assume that there are / = 1, A^ data points (X^, Y,) that 
suggest the use of the power function model (2.91a). The least-squares error reads 
for the model considered 

1\ ,=1 1\ ,=1 

= y,'-2y,{Y-a{X-x,)'') + ({Y-a{X-x,ff^ (2.92) 

= {y,-{Y) + a{u)y -{{Y)-a{u))\({Y-aUy). 

Here, we used in the second line the definitions (2.18) of means. In the third line 
we used the abbreviation U^ {X- Xq)*. It is assumed that Xq is given such that Xq ^ 
Xj for all i = I, N. This condition avoids problems with the existence of U for 
negative b values. If Xq has to be equal to one X^ value, then we may disregard this 
data point regarding the calculation of model parameters. In the following, the 
goal will be to find optimal y^, a, and b values that minimize E^. 



2.5 Optimal Power and Exponential Models 



65 



Optimal Jo- The optimal yg value is the value for which the non-negative first 
term on the right-hand side of Eq. (2.92) is equal to zero, this mean 



y,={Y)-a{U). 
The error can be written for this case 
{Y-aUy)-{{Y)-a{u)y 



Y' 



(YY -2a 



{{UY)-{U){Y)).ai{u^)-{uy 
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(2.94) 



The use of the variance definitions (2.23) enables a simpler writing ofE^, 
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(2.95) 
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Optimal a and b. The optimal a value is the value for which the non-negative 
second term on the right-hand side of Eq. (2.95) is equal to zero. Thus, a becomes 



UY 



a ■ 
The error E^ can be written then 



(2.96) 
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(2.97) 



Here, r^y refers to the correlation coefficient of U and Y. The optimal parameters 
Jo, «, and the error E^ obtained here correspond to the results for a linear function: 
see Eqs. (2.46) and (2.53). This is not surprising because the variables U, a, and 3/0 
used here correspond to X, a, and b in the linear function discussed in Sect. 2.3.2. 
The difference to the optimization of a linear function is that the error E^ given by 
Eq. (2.97) depends on the model parameter b. Unfortunately, there is no way to 
find the optimal b value analytically. Therefore, this value has to be determined 
numerically as the value for which E^ becomes minimal. The optimal parameters a 
and Jo can be obtained then on the basis of Eqs. (2.96) and (2.93). 



Table 2.1 Data used for the illustration of the optimization of a power fimction. 
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Fig. 2.11. An optimal power function model, (a) The error El <y^>"^ given by 
Eq. (2.97); this function has a minimum at Z? = -1.8312; (b) the dots show the 
{Xj, Y^ data values given in Table 2.1, the line shows the corresponding optimal 
power function model (2.98). 

Application. Let us illustrate the application of this approach. The data shown 
in Table 2. 1 represent a data set that can be described by a power function. The 
reference point Xq has to be unequal to the data points applied. For simplicity, we 
use Xq = 0. The error is/<y^>"^ determined by Eq. (2.97) is shown in Fig. 2.11a 
as a function of b. The minimum of this error is given ai b = -1.8312. For this 
value of Z? we find via Eqs. (2.96) and (2.93) the optimal values a = 9.4681 and 
Jo ^ 1.0237. The resulting power function model is then given by 

9.4681 



j^ =1.0237 + - 

X 

Fig. 2.11 shows that this model represents the data considered very well. 



(2.98) 



2. 5.2 Optimal Exponential Functions 



Error Definition. As a second example, let us address the optimization of the 
exponential function (2.91b). For this function, the least-squares error reads 



= jo^-2j.„(7-a(e''«--'-l)) + ((7-«(e*<-->-l)f) 
= {y^-{Y) + a{u)y-{{Y)-a{u))\{{Y-aUY 



(2.99) 



Here, we used the abbreviation {/= exp[b{X-XQ)] - 1. This error formula is equal 
to Eq. (2.92). Thus, the optimal values of jo, a, and b, and the resulting expression 
fori;^ are equal to the formulas given in Sect. 2.5.1. 
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Fig. 2.12. An optimal exponential function model, (a) The errorii/<y^>"^ given 
by Eq. (2.97); this function has a minimum 2Xh = 0.5043; (b) the dots show the 
data given in Table 2.2, the line shows the optimal exponential function model 
(2.100). 

Application. The following example illustrates the use of this approach. The 
data shown in Table 2.2 and Fig. 2.12 suggest the use of an exponential function 
for the modeling of these data. We may use the first point for the definition of 
Xo= 1. The error is/ < 7^ >"^ determined by Eq. (2.97) is shown in Fig. 2.12a. The 
minimum of this function is given at Z^ = 0.5043. The optimal values of a and jo 
are then given by a = 0.0963 and jo ^ 0.9959. With these model parameters we 
find the exponential model 



y^ =0.9959 + 0.0963 (e 



0.5043 (j-l) 



■!)• 



(2.100) 



The comparison of this function with the data in Fig. 2.12 shows that the model 
(2.100) accurately represents the data considered. 



2.5.3 Global Warming 

Global Temperature Modeling. Let us use now the power fimction modeling 
approach from Sect. 2.5.1 for finding an optimal model for real data: we will 
consider again the global temperature increase discussed in Chap. 1. The tempera- 
ture data are given in Table 1.7. The discussion in Sect. 1.5.2 showed that the 
development of the global temperature anomaly T in time t may be well described 



Table 2.2 Data used for the illustration of the optimization of an exponential function. 




X 1 2 3 4 5 6 7 8 9 


10 


Y 1.0 1.1 1.2 1.3 1.5 2.0 3.0 4.5 6.0 


10.0 
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Fig. 2.13. The global temperature increase, (a) The error E / <y*>"- given by 
(2.97); this function has a minimum at fo = 3.9402; (b) the dots show the data from 
Table 1.7, the solid line shows the optimal power function model (2.102). The 
dashed line shows the model (2.103) derived in Sect. 1.5.2. 



by the power function 

T = T,+a{t-Q''=T,+ 



t-t. 



(2.101) 



The last expression is written according to the result from Eq. (1.33) in Sect. 1.5.3. 
The parameter c introduced here is related to a and hhy c= \ I d^''. 

Optimal Model. The methodology for optimizing a power function described 
in Sect. 2.5.1 can be directly applied to this case: T, T^, t, and t^ correspond to y, 
Jo, X, and Xq, respectively. We set ?o " 1 840 in accordance with the conclusions of 
Sect. 1.5.2. The error E I <y^>"- is shown in Fig. 2.13a as a function of h. 
The minimum of this error is given ?Ab = 3.9402. Instead of the model parameter 
a we consider here c = \ I d^'', for which we find the optimal value c = 179.1856. 
The value Tq is found to be TJ) = -0.3339. These data result in the power function 

1840 



-0.3339 + 



t- 



179.1856 



(2.102) 



Application. The performance of the optimal model (2.102) is shown in Fig. 
2.13 in comparison to the data values. The model 



r = -0.35 + 



^-1840 
178 



(2.103) 



derived in Sect. 1.5.2 is also shown here. There is only a very minor difference 
between both models for the interval t = 1840 to ^ = 1870. Due to the scatter of 
data values, it is impossible to conclude that one of the two models is better than 
the other one: the performance of both models is good. The significant advantage 
of the optimal model (2.102) is that there is no information required in addition to 
the data values. On the other hand, the development of the model (2.103) obtained 
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before is not comparably simple. The modeling approach applied in Sect. 1.5.2 
works for this case, but it may not work for other cases. 



2.6 Summary 

This chapter addressed the question of how it is possible to find a model that 
does optimally agree with observations involving randomness. Let us summarize 
the conclusions regarding the questions considered in Sect. 2.1, this means the 
questions about the most convenient optimization concept, the difficulty of using 
this concept, and the benefits of the development of optimal models. 

Optimization Concept. There are many concepts available for minimizing the 
deviations between a model and data - so which concept should we apply? The 
optimization of linear relations represents the most important application area of 
optimization concepts. Therefore, various optimization concepts were compared 
for such a case (the linear noise model). Both the least-absolute-deviations error 
and Chebyshev error do not provide analytical expressions for the optimal model 
parameters. The Chebyshev error does not correctly describe the data trend, and 
the least-absolute-deviations error provides a curve that passes two data points 
(this means we end up with a result that can be obtained without using any optimi- 
zation). The least-squares error does not reveal such shortcomings. In addition, the 
use of this concept provides analytical expressions for the optimal parameters. 
Thus, only the least-squares error concept was used in the following. It is worth 
noting that this concept does not provide a tool for the evaluation of the perform- 
ance of models (we obtain an optimal model that may perform well or not). This 
question has to be addressed by calculating the relative error of model predictions 
(see Chap. 1). 

Ease of Use. How difficult is the use of the optimization concept? The point is 
that there would be not much interest in applying an optimization concept if its use 
turns out to be very complicated (we would consider alternatives). The application 
of the least-squares error concept for the optimization of linear functions (of rede- 
fined variables) is relatively simple. This concept can be also used for finding 
analytically optimal parameters for quadratic functions. The error concept can be 
also applied to the development of optimal power and exponential functions, 
although we have to numerically determine now the minimum of the error (the 
error is a function of one variable). Therefore, the error concept can be applied to 
all the basic functions considered in Fig. 1.17. The optimization concept can be 
also applied to more complex functions by determining optimal model parameter 
values numerically. However, such calculations are not simple anymore, in 
particular if we have to deal with three or more variables. 
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Benefits. The significant advantage of the least-squares optimization concept is 
that this concept usually provides appropriate model parameters in a very efficient 
way. There is no need to perform expensive studies of the effects of parameter 
variations, and there is no need for the consideration of averaged values, which 
were used in Chap. 1 to model the global temperature anomaly. The parameters 
obtained are at least as good as the results of specific investigations of effects of 
parameter variations. However, this optimization concept does not apply any other 
condition than the requirement to minimize the deviations between a model and 
observations. Thus, it may be the case that we obtain a nonzero vehicular stopping 
distance at a zero velocity. This concept does also not provide integer exponents 
that we prefer, for example regarding the presentation of Kepler's Third Law and 
the time dependence of the global temperature anomaly. Such questions can be 
addressed at best by adjusting the results for optimal model parameters according 
to the needs (by fixing integer exponents of power functions) and optimizing the 
remaining model parameters. 



2.7 Exercises 



2.2.1 Consider the data sets (d) and (f) given in Table 1.4. Assume that the data 
can be modeled in terms of the function j^^ = ax. 

a) Use the least-squares error, the Chebyshev error, and the least-absolute- 
deviations error to find the optimal value of a for the two data sets. 

b) Comment on the suitability of the three error concepts for the modeling 
of both data sets. 

2.2.2 Consider the data set (f) in Table 1.4. Assume that the data can be modeled 
in terms of the function jj^y = ax. 

a) Suggest two reasonable global error concepts in addition to the errors 
considered in exercise 2.2.1. 

b) Calculate the optimal value of a on the basis of your two global error 
concepts and the least-squares error. 

c) Comment on the suitability of your two error concepts. 

2.2.3 Consider the energy consumption data in Table 1.1. Assume that the data 
can be modeled in terms of the function C = at + h. 

a) Use the data point at 1950 to express Z? as a function of a. 

b) Calculate the optimal parameter a on the basis of the least-squares error, 
the Chebyshev error, and the least-absolute-deviations error. 

c) Graph the three curves. Comment on the suitability of error concepts. 



2.7 Exercises 71 



2.3.1 Consider the least-squares error E^ for a linear function yi^= ax + b (see 
Eq. (2.44) in Sect. 2.3.2), 

E^=lY-)-2a(xY) + a'{x-) + [b-{Y) + a{X))\ 

a) Determine directly the optimal b value that minimizes E^. Do not apply 
the Second Derivatives Test. 

b) Determine in the same way the optimal a value that minimizes E^. 

l.Ji.l Consider the quadratic model (1.29) for the atmospheric CO2 concentration 
development derived in Sect. 1.5.2, 

CO, =316 + 0.7(1 + ^~^^^^ |(?- 1959). 

a) Use the approach for the linearization of continuous functions described 
at the end of Sect. 2.3.2 for the development of a linear model for CO, 
for the period 1990-2010. 

b) Demonstrate the suitability of the linear model obtained by graphing 
both the quadratic model and the linear model. 

2.3.3 Consider the U.S. population data from 1790 to 1890 given in Table 1.5. 

a) Calculate the correlation coefficient r^^ for these (P, t) data. 

b) Consider the same t data. Replace the P data by the function values 
Pm= 0.0039 exp[(?- 1790)/35]. Find r^-for the {P,,, data. 

c) Use the data {P, t) from Table 1.5 to generate new (InP, t) data. 
Calculate the correlation coefficient r^ for the (InP, t) data. 

d) Replace in the latter data InP by \nP^,. Find r^^ for the (InP^,,, t) data. 

e) What do these four correlation coefficients tell us? 

2.3.4 Assume that this data set can be modeled by a power function j^ = a x'\ 
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a) Introduce new variables that are linearly related. Find the optimal model 
parameter values for this linear model by using the least-squares error. 

b) Graph the resulting optimal power function yu^ a x'' and the data. 

2.3.5 Assume that this data set can be modeled by a power function P = P^ + b f. 
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a) Introduce new variables that are linearly related. Find the optimal model 
parameter values for this linear model by using the least-squares error. 

b) Graph the resulting optimal power function P = P^^ + b f and the data. 
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2.3.6 Consider again the energy consumption data in Table 1.1. Assume that the 
data can be modeled in terms of the function C = at + b. 

a) Use the least squares error concept to find an optimal model. 

b) Graph the optimal model, the model C = 91 + 1.25 {t- 1995) obtained in 
Sect. 1.2.1, and the data. 

c) Comment on the performance of the optimal model in comparison to the 
model C= 91 + 1.25(^-1995). 

2.3.7 Consider again the U.S. population data from 1790 to 1890, which are 
given in Table 1.5. Assume that the data can be described by the exponen- 
tial fiinction P = c sxp{dt). 

a) Write the model as a linear model formulated in terms of new variables. 

b) Use the least squares error concept to find an optimal model. 

c) Graph the population data, the optimal model obtained in b), and the 
model P = 0.0039 exp[(^ - 1 790) / 3 5] obtained in Sect. 1.4.1. 

d) Comment on the performance of the optimal model in comparison to the 
model /* = 0.0039 exp[(?-1790)/35]. 

2.3.8 Consider again the U.S. population data from 1790 to 1890, which are 
given in Table 1.5. Assume that the data can be described by the logistic 
functionP = ^ / [1 + exp(C(?-5))]. 

a) Write the model as a linear model formulated in terms of new variables. 
Assume in consistency with the results of Sect. 1 .4. 1 that ^ = 0.12. 

b) Use the least squares error concept to find an optimal model. 

c) Graph the population data, the optimal model obtained in b), and the 
logistic model P = 0.12 / [1 + exp(- {t - 1890)/ 29)] obtained in Sect. 
1.4.1. 

d) Comment on the performance of the optimal model in comparison to the 
model P = 0.12 / [1 + exp(- {t- 1890)/29)]. 

2.3.9 Consider again the global temperature anomaly data given in Table 1.7 for 
every ten years (1850, 1860, ..., 2000). Assume that these temperature data 
can be described by the function T^A + [{t- 1840)/^]". 

a) Use the least-squares error concept to find an optimal model. 

b) Graph the resulting optimal model and the data. 

2.4.1 Consider the optimal quadratic model described in Sect. 2.4.1. 

a) What are the requirements for a data set considered such that the optimal 
quadratic model reduces to a linear optimal model? 

b) Show that the linear optimal model obtained by the simplification of the 
optimal quadratic model is equal to the linear optimal model derived in 
Sect. 2.3.2. 
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2.4.2 Consider the optimal quadratic model described in Sect. 2.4.1. 

a) Develop a numerical scheme for using the formulas in Sect. 2.4. 1 to find 
the parameters that minimize the least-squares error of this model. 

b) Calculate the optimal model parameters for the data set (0, 4), (1, 8.4), 
(2, 16.2), and (3, 26.3). 

2.4.3 Consider the total stopping distance data in exercise 1.3.1. 

a) Find the optimal quadratic model for this data set. 

b) Comment on the suitability of this optimal quadratic model. 

2.5.1 Consider the power function model Jm = Jo + « x*. Here, the exponent b is 
known, and the coefficients y^ and a have to be optimized. 

a) Determine the values of the parameters yg and a that minimize the least- 
squares error of this model. 

b) Assume that h = 2. Calculate the optimal model for the data set (0, 0), 
(1,3.4), (2, 14.1), and (3, 31.3). 

2.5.2 Consider the model j^,/ = ya + ln(x /x^)". Here, y^ and a are parameters that 
have to be optimized. Assume that Xq is known. 

a) Determine the values of the parameters y^ and a that minimize the least- 
squares error of this model. 

b) Calculate the optimal values of Jq and a for the data set (1, 0), (2, 3.1), 
(3, 4.9), and (4, 6.2). Assume that Xq = 1. 

2.5.3 Consider the model jj^ = Jq + « ^* s^- The parameters y,^, a, and b have to be 
optimized. 

a) Reduce the problem such that the least-squares error does only depend 
on one variable. Hint: follow the approach presented in section 2.5.1. 

b) Explain the use of these formulas for the calculation ofy^, a, and b. 

2.5.4 Both the exponential function model >'j^ = jo + «(exp[fo(x-Xo)]- 1) and the 
power function model Jm ^ 3^o + « (^ ~ ^o)* require the setting of x^. The 
parameters j'o, a, and b can be optimized as explained in Sect. 2.5. 

a) Can we have the same optimal exponential function model for two dif- 
ferent choices of Xq? Show the parameter relations if the answer is yes. 

b) Can we have the same optimal power function model for two different 
choices of Xq? Show the parameter relations if the answer is yes. 

2.5.5 Consider the exponential function model j'M^J'o + fl (exp[Z7(x-Xo)]- 1). 
The parameters y^,, a, and b have to be optimized. 

a) Develop a numerical scheme for using the formulas in Sect. 2.5.2 to find 
the j'o, fl, andZ? values that minimize the least-squares error of this model. 

b) Calculate the optimal jvo, «, and b for the data set (1, 1.95), (2, 1.91), 
(3, 1.87), and (4, 1.84). Use Xj = 0. 
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2.5.6 Consider the power function model y^^ = y^ + a {x - Xq)''. The parameters 
Jo, a, and b have to be optimized. 

a) Develop a numerical scheme for using the formulas in Sect. 2.5.1 to find 
the parameters y^, a, and b that minimize the least-squares error of this 
model. 

b) Calculate the optimal jvo, a, and b for the data set (1, 9), (2, 2), (3, 1.3), 
and (4, 1.1). Usexj = 0. 

2.5.7 Consider again the exercise 2.5.6. 

a) Redo the calculation for Xq = 0.5. 

b) Redo the calculation for Xg = 0.8. 

c) Calculate the maximum relative error for the power fiinction models that 
apply Xq = 0, Xq = 0.5, and Xg = 0.8. Which of the three Xq considered is 
the best choice? 

2.5.8 Continue with the exercises 2.5.6 and 2.5.7. 

a) Present at least two ideas of how it is possible to design a model that 
performs better than the three models considered in exercises 2.5.7. 

b) Demonstrate the suitability of your approach by designing a model that 
performs better than the three models considered in exercises 2.5.7. 



3 Deterministic States 



The deterministic methods discussed in Chap. 1 were presented for problems 
that require the modehng of observations of one variable (like the total stopping 
distance of cars, the energy consumption, the atmospheric COj concentration, or 
the global temperature anomaly). However, most real problems are characterized 
by relations that involve several variables. Such relations have to provide the 
correct dimension of variables considered (for example, a characteristic time scale 
of any problem has to be calculated by a relation that provides a time). This 
constraint implies a reduction of all possible relations between variables, i.e., it 
reduces the original complexity of problems significantly (for example, it may 
require that a certain variable cannot be involved in a relation). The technique that 
provides such a problem reduction will be described in this chapter. This approach 
is extremely helpful, but it cannot completely solve the problem. The equations 
obtained in this way still involve unknown parameters that have to be determined 
by means of observations. Hence, the methods to be described in this chapter 
usually represent the first step before using the modeling approaches presented in 
Chap. 1 to determine unknown parameters. 

Section 3.1 illustrates the need for the development of a method for the analysis 
of problems that involve several variables. Section 3.2 introduces the theoretical 
basis for the reduction of several-variable problems, i.e., dimensional analysis. 
The relevant concept of similarity is also explained. This concept provides the the- 
oretical basis for one of the most important applications of dimensional analysis: 
the possibility to study the properties of real systems (e.g., the forces on an 
aircraft) by means of much more efficient investigations of little model systems. 
Applications of these techniques to the solution of problems of low complexity, 
medium complexity, and relatively high complexity will be described in Sects. 
3.3, 3.4, and 3.5, respectively. Section 3.6 summarizes the basic features, advan- 
tages, and limitations of the approach presented. 
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Fig. 3.1. An illustration of factors that affect the lift force of aircrafts. 

3.1 Motivation 

Lift. Let us consider aircraft flight as an example to illustrate the need to deal 
with problems that involve many variables. In particular, let us try to find a 
formula for the dependence of the lift force F[^ on various factors. The lift force Fi^ 
is a force that is generated to overcome the aircraft weight, i.e., Fj^ is the force that 
makes the aircraft fly. Figure 3.1 shows an illustration of how the lift force is 
generated and affected by various factors. The airflow passing under the wing is 
deflected downward by the bottom surface of the wing. The downward deflected 
air is pushing against the surrounding air, the surrounding air is stopping the air 
deflected downward and pushing this air upward (this process is similar to the 
flight of a balloon: if air is rushing out of a balloon, the reaction is that the balloon 
is forced away). The more air is deflected downward, the more lift will be created. 
The theoretical explanation for this process is provided by Newton's Third Law - 
for every action there is an equal and opposite reaction (Newton's Laws will be 
presented in Chap. 7). 

Lift Influence Factors. There are several factors that may affect the magnitude 
of the lift force 7^^: see the illustration in Fig. 3.1. The lift force will depend on the 
wing geometry. The wing shape may be characterized by two length scales Ji and 
fi?2 (which is called the chord length). The lift force will also depend on the relative 
aircraft velocity v and the angle of attack 0. The relative aircraft velocity v is the 
speed of an aircraft relative to the airmass in which it flies, i.e., v is the magnitude 
of the vector difference between the aircraft velocity and the air velocity. The 
angle of attack d refers to the angle of inclination of the wing. The lift force will 
also depend on the properties of the fluid considered: the air density p, the air 
viscosity //, and the speed of sound s. There is one additional factor that may 
affect the lift force: gravity - which acts in the opposite direction of the lift force. 
The parameter that accounts for the gravity force is the gravity acceleration g. By 
extending the analysis presented below by the inclusion of the parameter g one 
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can prove that gravity may contribute to 7^^ via the ratios (t/j g)"* / u or (o?, g)"^ I v. 
To assess the relevance of these ratios let us consider t/j ^ 10 m (the value of c/j is 
smaller than d^}. The latter assumption results in the inequality ((i2g^)"^ - 20 km/h. 
Most airplanes fly at much higher velocities. Thus, the ratio {d2gY^ I v will be 
negligibly small such that we can disregard the effect of g. Correspondingly, we 
expect F[^ as a function of seven variables, 

F,-=Fi^{v,di,d2,0,p,p,s). (3.1) 

Experimental Studies. Let us consider the use of measurements for the 
evaluation of the lift force in dependence of these seven variables. One way to 
address this problem is to measure the dependence of 7^^ on one parameter, where 
the other six parameters are kept constant. To have an idea about the number of 
measurements required, let us consider the case that we need five data points 
(experiments) to characterize the influence of one parameter on 7^^. To study the 
dependence of 7^^ on a second variable, we need 5" measurements. The evaluation 
of the dependence of 7''^ on 7 parameters then requires 5^ = 78,125 experiments! 
Obviously, the costs of such a number of experiments wiU be huge. Apart from 
that, five measurements are not always sufficient to accurately determine data 
trends, and there will be also a significant influence of randomness in such data. 

Problem Considered. The discussion in the preceding paragraph showed that 
experiments cannot be used directly in general to determine the dependence of 
variables like Fj^ on a relatively large number of factors. A first essential step is 
needed: we have to reduce the number of influence factors as much as possible. 
Experiments, theoretical results, or simulation results can be used then to study the 
detailed properties of reduced equations. 



3.2 Dimensional Analysis and Similarity 

A way to address the problems described in the previous section is given by the 
application of dimensional analysis, i.e., the analysis of implications of the most 
basic property of variables to have a dimension. Let us see how this works. 



3.2.1 Buckingham's Theorem 

Dimension. We are interested in variables that can be observed, i.e., we are 
interested in measurable variables. The most basic property of such variables is 
that measurable variables have a dimension. For example, the dimension of the hft 
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Table 3.1 Dimensions of physical variables in the LMT system. 



Variable 



Dimension 



Variable 



Dimension 



Mass 


M 


Work 


ML'r^ 


Length 


L 


Pressure 


ML-' r' 


Time 


T 


Power 


ML^r^ 


Frequency 


r' 


Angle 


AfL'f 


Velocity 


ir' 


Velocity of sound 


Lr' 


Acceleration 


Lr'- 


Density 


ML-"" 


Force 


Mir^ 


Dynamic viscosity 


ML-' r' 


Energy 


mlT^ 


Kinematic viscosity 


L'r' 



force F[^ considered above is mass times acceleration, this means mass times 
length divided by a squared time. Table 3.1 shows examples for the dimension of 
other variables. To work with dimensions of variables we need a mathematical 
formulation of dimensions. Dimensions are represented by a certain unit system. 
One unit system is given by the LMT system. Here, L refers to any unit length L, 
M refers to any unit mass, and T refers to any unit time. Examples are given by 
Z = 1 m, M = 1 g, and T = 1 s. Another example is the choice Z = 1 km, M= \ kg, 
and T= 10^ s. The definition of such units is a requirement for measurements. To 
measure any variable actually means to compare the variable directly or indirectly 
with an appropriate standard (with a certain unit of the measurement). Here, L, M, 
and r serve as units such that every measurement of a length, mass, or time can be 
represented as a multiple of the corresponding unit. Dimensions of variables are 
then given as certain combinations of units. For example, the dimension of the lift 
force Fi considered above is given by M Z 7"^. The LMT system can be used to 
study a wide range of problems in mechanics. Nevertheless, the ZAZr system is not 
general in the sense that it can be used for all problems. Additional standards have 
to be taken into account if problems are studied that involve, for example, heat 
transfer or electromagnetic systems. However, we will consider only the LMT 
system in the following. This sort of analysis is adequate to explain all the basic 
features of dimensional analysis. Additional properties can be easily taken into 
account if required. 

Dimensionally Correct Equation. The most basic constraint that we can use 
for the calculation of variables is that the formula applied must have the correct 
dimension of the variable considered: formulas have to be dimensionally correct. 
Let us illustrate the benefits of this constraint regarding the calculation of the lift 
force Fi = Fiiv, d^ dj, 0, p, jU, s) as a function of all these parameters. We know 
that F[^ has the dimension of mass times acceleration, i.e. 

ML 



Fl 



(3.2) 
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To relate the right-hand side to variables involved in this problem we rewrite 
Eq. (3.2) in the following way: 

The first term Ml 1} is written in terms of the dimension of the air density p. This 
term accounts for the influence of M on Fi. The second term 1} I f^ is written in 
terms of the dimension of v^. This terms accounts for the influence of T on Fi. The 
third term L^ is written in terms of the dimension of d^ (another possible choice 
would be the use oid^ instead oid^. This term accounts for the effect of Z, on F^^. 
According to Eq. (3.3), we may expect the following formula for the lift force F^ 

F,=^pv'd,\ (3.4) 

Here, Q is the so-called lift coefficient. The inclusion of the factor 1/2 in Eq. (3.4) 
corresponds to the general practice of writing this formula. The variable/) = pv^ll 
is called the dynamic pressure. Thus, Eq. (3.4) can also be written F^^ Cip d2- 
Equation (3.4) includes only a few variables of the list of all possible variables 
that may affect F^. Does this mean that the other variables are irrelevant? The 
latter conclusion is incorrect because the other variables may affect F^ via Q. 
However, Q is a nondimensional number, this means Q is independent of M, T, 
and L. A closer look at the other variables shows that Q can be, for example, a 
function like 



Q=Q 






(3.5) 



The star indicates that Q may depend on additional nondimensional parameters. 
An important conclusion at this point is that a dimensionally correct equation for 
Fl can be written as a relation between nondimensional products. 



pv'd,' 2 ' 



^2 S 



(3.6) 



where Eq. (3.4) was applied. In other words, only equations that can be written in 
terms of nondimensional products are dimensionally correct. Equation (3.6) can 
be written in a more general way. Let us introduce the following abbreviations for 
the nondimensional products involved in Eq. (3.6), 



PV d^ Mj 
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By using these abbreviations, Eq. (3.6) can be written 

P,=f{P„P„P„-). (3.8) 

Buckingham's Theorem. Equation (3.8) does not only apply to the lift force 
problem considered here. It can be used for any nondimensional products, and it 
may involve many additional nondimensional products like P^, P^, ■■■. Equa- 
tion (3.8) is called Buckingham's Theorem (Buckingham 1914). This theorem 
states the following. Every dimensionally correct equation can be written like Eq. 
(3.8): a dimensionally correct equation represents a relation between nondimen- 
sional products. A dimensionally correct equation is also called a dimensionally 
homogeneous equation. Such equations are correct regardless of the system of 
units in which the variables are measured. The nondimensional products involved 
in Eq. (3.8) are considered to be independent, this means no nondimensional 
product can be represented by a combination of other nondimensional products. 
The nondimensional products involved are considered to represent a complete set 
of nondimensional products. This assumption means that the nondimensional 
products are independent, and that all possible nondimensional products can be 
obtained through combinations of these products. A good way to see whether a 
particular set of nondimensional products is complete or not is to ask whether all 
the relevant variables are involved, and whether every product involves one 
physical variable that does not appear in any other nondimensional product. 

Open Question. The relevance of Buckingham's theorem is that we cannot 
work with any equations, but only some equations (which are in consistency with 
Buckingham's theorem) are correct. The reduction of the number of equations that 
can be used implies a significant simplification of problems. However, Bucking- 
ham's Theorem does not explain how we can derive the most general nondimen- 
sional equation for any specific problem: this theorem does only tell us that we 
have to relate nondimensional products. Thus, there is the question of how it is 
possible to determine all nondimensional products that should be considered in a 
nondimensional equation for a given problem. 



3.2.2 Dimensional Analysis 

Dimensional Analysis. Let us address the latter question regarding the lift 
force problem (the same approach can be applied to other problems: see below). 
We consider the following general product involving variables that may affect 7^^, 

F/v'd.'d/d'p' ju' s". (3.9) 



3.2 Dimensional Analysis and Similarity 



Here, the exponents a, b, c, d, e, /, g, and h are any unknown (positive, negative, 
or zero) real numbers. The condition for having a nondimensional product is then 

F," v" d^' d^' 9' p' Ij' s" =c^. (3.10) 

The symbol Cj on the right-hand side refers to any constant that is independent of 
L, M, and T. To understand the implications of Eq. (3.10) we have to consider the 
dimensions of the variables involved. By replacing the variables by their corre- 
sponding dimensions given in Table 3.1, Eq. (3.10) can be written 

(^J[fJwW(i"«.r.)-(^J(iLj(iJ.c.. ,3.n, 

Here, C2 is a constant that is independent of Z, M, and Tand different from c,. The 
angle 9 is independent of Z, M, and T. Therefore, 9 is replaced by Z" A/* 7* here. 
Equation (3.11) can be written in a more convenient way by combining terms that 
involve Z, M, and T, 

Ta+b+c+d-3f-g+h ,ta+f+g -p-la-b-g-h _ ^3 12") 

This condition is satisfied if the exponents of Z, M, and T disappear, this means if 

= a+b + c + d-3f-g + h, (3.13a) 

= a + f + g, (3.13b) 

0^-2a-b-g-h. (3.13c) 

Equations (3.13) can be solved in different ways. Here, we use these equations for 
the calculation oi b, d, and/ this means b, d, and /are considered as dependent 
variables, and a, c, e, g, and h represent independent variables. Equations (3.13b) 
and (3.13c) can be used for the calculation of/ and b as functions of independent 
variables, but Eq. (3.13 a) also involves the dependent variables /and b. Thus, we 
rewrite Eq. (3.13 a) by means of the expressions for /and b that are provided by 
Eqs. (3.13b) and (3.13c), 

= a-{2a + g + h)+c + d + 3{a + g)-g + h = 2a+c + d + g. (3.14) 

Correspondingly, b, d, and/can be calculated by the equations 

d = -2a-c- g, (3.15a) 

f = -a-g, (3.15b) 

b = -2a-g-h. (3.15c) 

The use of these expressions in Eq. (3.10) then provides 

F/ u-"-'-" d; di^"'"'' 9" p-"-^ n^ s" = c,. (3.16) 
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The latter relation can be also written 
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(3.17) 



Buckingham's Theorem. This relation is the general condition for having a 
nondimensional product. It identifies five numbers 



pv^ d. 



0, 



/" 



pd-^v 



(3.18) 



All these numbers have to be independent of Z, M, and Tto satisfy Eq. (3.17) for 
all a, c, e, g, and h. Hence, we found five nondimensional products. The latter fact 
can be easily proven by using the dimensions of these variables in each of these 
products. The nondimensional products obtained are independent because each 
product contains a variable (F^, 0, di, jU, s, respectively) that does not appear in 
any other product. The nondimensional products (3.18) represent a complete list 
of nondimensional products because all variables considered are involved, and the 
products (3.18) are independent. Therefore, Eq. (3.18) provides all the non- 
dimensional products that we need to find the general nondimensional equation 
for the lift force F^^. According to Buckingham's Theorem, this nondimensional 
equation is given by 



/ 



-,o^ 



M 



pv d-, fi?2 ^ yOfi?,^ 



= 0. 



By assuming that we can solve for FJ(pv^ d,), Eq. (3.19) can be written 



where the lift coefficient Q is given by 

f d „ n ^ 



M 



d, s pd^v 



(3.19) 



(3.20) 



(3.21) 



This result is the most general expression for Q. Equation (3.5) is generalized by 
this expression for Q by the consideration of // / {p d, v). The derivation of the 
structure of the lift force formula in this way represents a significant advantage 
compared to the consideration of the lift force F^ = Fi{v, di, d,, 9, p, //, s) as a 
function of seven variables. 

First Variation. Expression (3.20) for F^ combined with the lift coefficient 
(3.21) is not the only result that can be obtained by the application of dimensional 
analysis. In order to illustrate the possible variations, let us consider b, c, and/as 
dependent variables instead oi h, d, and /used before. According to Eq. (3.15), the 
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equations for b, c, and/are given by 

c = -2a - d - g, 

f = -a-g, 

b = -2a — g — h. 

The use of the latter relations in Eq. (3.10) provides 



-g-h ^ -i"-d-S J^rf ge p-c-s ^g ^h ^ 
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(3.22a) 
(3.22b) 
(3.22c) 



(3.23) 



= c,. 



For this case, the corresponding list of nondimensional products is given by 
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Instead of Eq. (3.20) we obtain then 



C, 



F, =^^pv^d^^, 



(3.24) 



(3.25) 



where the lift coefficient Q is given by 



Q=Q 



d^ V n 



fi?, s pdyV 



(3.26) 



Second Variation. Let us consider a second variation where a, b, and/are the 
dependent parameters. Equations (3.15) show that a, b, and/are determined by 



a = --{c + d + g), 

f = -a-g = -{c + d-g), 

b = -2a — g — h = c + d — h. 



(3.27a) 
(3.27b) 
(3.27c) 



Here, Eq. (3.27a) was used to write the other two relations as functions of 
independent variables. The use of these relations in Eq. (3.10) then provides 



F,-''^'^'"'v'''-"d;d/d'p 



{c+d-g)/2 g ^h 
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(3.28) 



= c,. 



The corresponding list of nondimensional products is given for this case by 



pv^ d^ 
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pv rfj 



e. 






(3.29) 
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This list of nondimensional products cannot be used to represent F^ as a function 
of other parameters because 7^^ appears in three of the products in Eq. (3.29). 

Observations. Let us summarize the observations regarding the variations of 
analysis results described in the preceding three paragraphs. 

• A closer look at the influence of different choices of independent and dependent 
variables shows that the choice of an independent variable implies that this 
variable appears in only one nondimensional product, whereas the choice of a 
dependent variable implies that this variable can be found in many nondimen- 
sional products. For example, for our last case considered we used c, d, e, g, h as 
independent variables and a, b, and /as dependent variables. The variables di, 
d2, 0, jU, s related to c, d, e, g, h appear only in one nondimensional product, 
whereas the variables F^^, v, p related to a, b, and /are found in several non- 
dimensional products. 

• There are many different ways to write the results of nondimensional analysis 
depending on the choice of dependent variables: new nondimensional products 
can be obtained by using different dependent variables. These differences do not 
pose any problem as long as the exponent related to the variable F^ (which we 
like to calculate) is taken as an independent variable. Otherwise, Fi is found in 
many nondimensional products. 

• A procedure for obtaining new products in a simple way is to consider powers of 
given nondimensional products, or to multiply or divide nondimensional pro- 
ducts with (powers of) other nondimensional products; e.g., the operations 
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(3.30) 



enable the transition from the list (3.18) to the list (3.24.) 



3.23 Similarity 



The Problem. A usual problem, which will be illustrated again regarding the 
lift force problem, is the following one. After applying dimensional analysis, we 
found for 7^^ the formula 
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(3.31) 



where the lift coefficient Q is given by the function 



Q-Q 
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(3.32) 
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Here, the kinematic viscosity v = jj I p k used to simplify the writing. The lift 
coefficient Q is an unknown function: we do only know that Q depends on four 
parameters. What we need to know is the value of Q for any specific conditions 
[e.g., Q(10, 0.1, 0.4, 10"), see Sect. 3.5], to be able to calculate the lift force 7^^ 
for this case. Let us call this value 



c/=c/ 



( J R R R \ 

' J R 'R ' J R R 
«2 ^ d2 V J 



(3.33) 



Here, the superscript R refers to the conditions (the values of variables) of the real 
system. Usually, investigations of Q for the real system (the aircraft) are way too 
complicated and expensive. Thus, we try to use a little model system to determine 
Q. In the little system we measure, e.g., Q(10, 0.1, 0.004, 10 *"), or, more general 
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J M ' U ^ J R U 



(3.34) 



Here, the superscript M refers to the conditions (e.g., the length scales d^ and c/,*') 
of the model system. Do we calculate the lift force for the real system correctly if 
we use such a measured C^* in the lift force formula (3.31)? This approach will 
provide an incorrect result, in general, because of the following. We need the 
function Q at the ^-values [e.g., ^(10, 0.1, 0.4, 10 ')], but we provide Q at the 
M-values [e.g., Ci(10, 0.1, 0.004, 10'')], which will be different from the R- 
values, in general. This means, we consider Q at the wrong function values. 

The Solution. The way to overcome the latter problem is to measure Q in the 
model system at the values required for the real system, this means we have to 
make sure that all M-ratios are equal to the corresponding 7?-ratios, 

fl, 02 s s d^ V dj V 

For this case of equal arguments of Q we have 

C," -C,\ (3.36) 

which means we have the Q value that is required in the lift force formula (3.31). 
The last two relations show that all nondimensional products involved have to be 
equal in the model and in the real system. Such systems are called similar systems. 
It is worth noting that the similarity of systems does not only require the similarity 
of geometries (equal 9 and di I d^) - it requires the equality of all nondimensional 
products involved in the problem considered. Equations (3.35) represent the 
design conditions for the development of the model system. The design of the 
model has to be performed such that O'''^ 9^ and {d-^l d^f' = (d^/ d2)'' are satisfied. 
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Fig. 3.2. Air and water properties, (a) and (c) show the kinematic viscosity v and 
speed of sound s of air and water in dependence on the temperature T (in °C); (b) 
and (d) show the ratios v^,,./ v,|/„,„. and s^,^,„.l SJ^i,, respectively. 

The latter is not very complicated, in general. It means that the model has to look 
like the real system. Written as conditions for the velocity ratio x^' I u", the last two 
conditions of Eq. (3.35) that have to be satisfied are given by 



v" J/ 



(3.37) 



In essence, it is not simple at all to satisfy these two conditions. By replacing the 
velocity ratio in the second relation according to the first relation, these conditions 
can be also written as 



v^ s'' 
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(3.38) 



The second condition shows that it needs another fluid to obtain a model that is 
smaller than the real system. 

Kinematic Viscosity and Speed of Sound of Air and Water. Let us consider 
the kinematic viscosity v and the speed of sound s of air and water to prepare the 
use of these properties for the discussion of the realization of design conditions. 
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The viscosity of air at standard atmospheric pressure is given (in m^/s) by 

v^,,. =3.448410"'(-0.0683«'+2.0712«'+2.9786«-l). (3.39) 

Here, a = {T + 273.15) / 273.15, and T refers to the temperature in Celsius. This 
formula represents a polynomial curve fit to a data set that is given in the SFPE 
Handbook of Fire Protection Engineering (1995). The range of validity of this 
formula is 100°K <T< 1600°K. Due to thermodynamics (Heinz 2003), the speed 
of sound (in m/s) in dry air at standard atmospheric pressure is 

s^„.= 231.5 4a. (3.40) 

This formula can be used at least for -40°C < T < lOO^C. According to Chenlo 
et al. (2004), the viscosity of water is given (in m^/s) by 

^r«,.. = 0-09607 10-V'"'. (3.41) 

This fonnula can be applied for 0°C <T < 100°C. The speed of sound (in m/s) in 
water can be described by (Del Grosso & Mader 1972, Lubbers & Graaff 1998) 

%.to=24,130(0.1983«'-1.3322«^+3.6287«'-5.0548«H3.6182«-l). 

(3.42) 
This relation is applicable for 0°C <T < 100°C. The dependence of vand s on the 
temperature T is illustrated in Fig. 3.2. The plot of the corresponding ratios in 
Figs. 3.2b and 3. 2d shows that in particular the viscosities of air and water differ 
significantly. It is worth noting that the corresponding ratio of dynamic viscosities 
is very different to v^,,./ Vf,,^,^,. shown in Fig. 3.2 because we have //^„. < //^ato- 

Realization of Design Conditions. Let us consider now the question of how it 
is possible to satisfy the design conditions (3.38). For doing this we consider the 
following case. In reality, we have to consider air at a temperature T = -40°C. 
Hence, we have v'^^ 10.3760 10 " m^ / s and and / = 306.2673 m / s. To end up 
with a model that is relatively small we need a model viscosity that is relatively 
small: see the second condition (3.38). Let us say we use water at a temperature 
T= 40°C. Then, we have v" = 0.6583 10 ' mV s and and Z' = 1532.5990 m / s. 
The design conditions (3.38) read for these values 

V = 5.0041, ^ = . (3.43) 

v" d/ 78.8737 

Hence, a complete similarity can be achieved by a model system that is about 79 
times smaller than the real system and a model velocity that is five times higher 
than the real velocity. It may be the case that there is no way to realize a model 
velocity that is higher than the real velocity. This problem can be solved if the 
aircraft velocity is much smaller than the velocity of sound s" = 306.2673 m/s. 
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For this case, the hft coefficient can be considered to be independent of s. Then, 
the design conditions (3.37) in addition to the conditions d''' = (9" and {di I d^'^ = 
{djd^'^ reduce to only one condition, 

^ = ^^ (3 44) 

For the conditions considered, v" = 10.3760 10 '^ mVs and v*'= 0.6583 10"^ m'/s, 
this relation is given by 



v"" _ 1 d/ 



v" 15.7618 1// 



(3.45) 



For the condition v'^'=v'^ that we can realize we find d-^ « dj^ I 16, this means the 
model system has to be about 16 times smaller than the real system. 



3.3 Applications of Low-Complexity 

We will consider three examples to illustrate the use of dimensional analysis: 
the vehicular stopping distance and Kepler's Third Law, which were considered in 
Chap. 1, and Stokes' Law, which will be applied in Chap. 6. The problems 
considered here have a low complexity, which means that the nondimensional 
equations to be obtained will involve only one nondimensional product. 



3.3.1 Vehicular Stopping Distance 

Two Processes. Let us consider the use of dimensional analysis regarding the 
modeling of the total vehicular stopping distance that was discussed in Sect. 1.2.3. 
We have to define first the variables that determine this problem. There are two 
processes that contribute to the total stopping distance D, 

D = D^+Ds. (3.46) 

The first contribution Dp, refers to the reaction distance. The reaction distance is 
the vehicular travel distance between two events: the moment at which the driver 
realizes the need to stop the vehicle, and the moment at which the brakes are 
actually applied. The second contribution to the total stopping distance D is the 
braking distance Dg. The braking distance is the vehicular travel distance between 
the moment at which the brakes are actually applied, and the moment at which the 
vehicle stops. Let us separately analyze these two processes. 
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Reaction Distance. Which variables will determine the reaction distance? 
First, the reaction distance will depend on the driver's reaction time 7"^: the faster 
the reaction time, the smaller will be the reaction distance. Second, the reaction 
distance will depend on the vehicular velocity v. the higher the velocity, the larger 
will be the reaction distance. Therefore, we can expect the reaction distance as a 
function Z)^ = Dif(Tf,, v). The dimensional constraint reads then 

Z)/r/i;^ = c„ (3.47) 

where Ci is any constant that is independent of Z, M, and T. By accounting for the 
dimensions of Z)^, Tg, and v, Eq. (3.47) implies that 

-i/; I ^ I ja+c rph-c 



rr - = L"^' r-' = c,_, (3.48) 

where Ci is a constant. The condition that the left-hand side is independent of Z, 
M, and T then leads to the following two conditions (there is no third condition 
because M is not involved here), 

= a + c, (3.49a) 

= b-c. (3.49b) 

We use the exponent a as an independent variable because we would like to solve 
for Z)^. Hence, c = - a, and b = c = - a. Equation (3.47) can be written then 



^ D ^ 

\TrVj 



= c,. (3.50) 



We obtain, therefore, only one nondimensional product D^ I (Tg v). This product 
must be constant because there is no other nondimensional product that can affect 
Dnl{T^ v). Correspondingly, 

D,=c,T,v, (3.51) 

where the constant c^ is introduced. We may set c^ = 1. In this case, the formula 
obtained represents the usual formula for the distance traveled by a body that 
moves with a constant velocity, 

D,=T,v. (3.52) 

Brali^ing Distance. Which variables will determine the braking distance Dgl 
The braking distance is determined by the brake force Fg applied. It will also 
depend on the vehicular velocity v. the higher the velocity, the longer will be the 
braking distance. The mass m of the vehicle considered will also influence the 
braking distance: the higher the vehicular mass, the longer will it take to stop. 
Therefore, we can expect the braking distance as the function Dg = Dg{Fg, v, m). 



90 3 Deterministic States 



We set up the constraint for the dimensional analysis, 

D; Fj'v'm'' = c„ (3.53) 

where Cj is any constant. By accounting for the dimensions of Dg, Fg, v, and m, 
the latter condition leads to 

L" f^l (k)M'= Z""*^'- M'^' T-''-' = c„ (3.54) 

where C2 is a constant. The condition that the left-hand side is independent of Z, 
M, and Tthen implies three conditions: 

= a + b + c, (3.55a) 

= b + d, (3.55b) 

= -2h-c. (3.55c) 

For deriving a formula for Dg we use the parameter a as an independent variable. 
The latter two relations imply b = -c 1 2 = - d, such that the first relation implies 
a = -b + 2b = b. Correspondingly, there are the following three conditions for the 
dependent variables b, c, and d, 

b = a, (3.56a) 

d = -a. (3.56b) 

c = -2a. (3.56c) 

Equation (3.53) can be written then 



'F,D,^ 






m 



mv^ 



(3.57) 



The result of the dimensional analysis is, therefore, one nondimensional product 
FgDgl{m v^), which must be constant, i.e. 

Ds=cg!^. (3.58) 

The constant Cg introduced here can be calculated by considering the energy 
balance. The work needed to reduce the kinetic energy mv^/2 of the car to zero is 
given by Fg Dg. In particular, the work Fg Dg has to balance the kinetic energy 
mv' 1 2, this means FgDg = mv^ 12. A look at Eq. (3.58) reveals that this require- 
ment implies Cg = 1/2. Hence, the braking distance reads 

Dg=^—. (3.59) 

Here, the constant braking acceleration gg = Fgl m is introduced. 
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Comparison with Experiments. Accordingly, the total stopping distance 
D^D/i + Dgis found to be given by 



D- 



f \ 



(3.60) 



The comparison with Eq. (1.9) derived from experimental data, 



D-[^-Y,r^ (3-61) 

reveals the validity of the conclusion (3.60) obtained here: D is the sum of a linear 
and a quadratic function of u. The comparison of Eqs. (3.60) and (3.61) enables 
the calculation of the unknown parameters in Eq. (3.60). In Eq. (3.61), the total 
stopping distance D is measured in feet, and the vehicular velocity v is measured 
in mph. Hence, Tp. = 2 fil mph and g^ = 10 mph^ / ft. By relating ft to m (1 ft = 
0.3048 m) and mph to m/s (1 mph = 0.4470 m/s), the parameter values obtained 
are given by 7"^ = 1.36 s and gg = 6.56 m/ s^. A mean reaction time T^ = 1.36 s 
appears to be a reasonable value. A braking acceleration gg equal to 67% of the 
gravity acceleration is also a plausible value. 



3.3.2 Kepler's Third Law 

Dimensional Analysis. Next, let us consider Kepler's Third Law discussed in 
Sect. 1.2.2 as another example. What will be the relevant variables for the calcula- 
tion of the orbital period Tpl Definitely, the orbital period will depend on the 
mean distance r from the Sun. Tp will also depend on the gravity force F^ and the 
mass m of the planet considered. Thus, we expect a function Tp = Tpir, Fq, m) for 
the orbital period. To set up the dimensional analysis we use the condition 

Tp" r Fq m =Ci, (3.62) 

where Ci is any constant. By accounting for the dimensions of Tp, r, Fq, and m, 
this condition reads 

,( ML 



na jh\ -'" ^ \ ji jf d jh+c ][yfC+d rpa—lc 



r U I -^ I M" = U*' M'*" T'-" = Cj, (3.63) 



where Ci is any constant. This condition is satisfied if 

= Z> + c, (3.64a) 

= c + d, (3.64b) 

= a-2c. (3.64c) 
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We use the parameter a as the independent variable. The latter three conditions 
provide then for h, c, and d the constraints 

b = -a/2, (3.65a) 

d = -a/2, (3.65b) 

c = a/2. (3.65c) 

Hence, Eq. (3.62) can be written 

= Ci. (3.66) 



rj-i a -a/2 T7 ^^2 -all 

Ip r Pq m 



FcTp 



V """ y 




Dimensional analysis reveals, therefore, that there is only one nondimensional 
product Fc Tp /{m r), which must be a constant. Correspondingly, we can have 



(3.67) 



Here, a constant 2 ncp is introduced, where Cp is unknown. This formula does not 
involve details of the elliptical path of planets (only the mean distance r from the 
Sun is involved). Therefore, we may assume that the planets revolve around the 
Sun on a circular orbit. According to the laws of mechanics we have Fq = m a„ 
where the circular acceleration a^ = 4 7i^ r / Tp. The comparison of the resulting 
FQ = An^rmlTp with (3.67) shows that Cp=\. Hence, the result of our dimen- 
sional analysis reads 

(3.68) 

Here, the appearance of 2 ;r corresponds to the usual relation between any period T 
and the related angular frequency m^lnl T. Hence, Eq. (3.68) defines the related 
angular frequency as cOp = [7^^/ (m r)]"^. 

Comparison with Experiments. Let us compare the latter formula with Eq. 
(1.8) obtained by the analysis of experimental data. 





Tp=2n\ — . (3.69) 



Here, G^ = 1.3291 x 10^" mVs^ refers to the standard gravitational parameter. Both 
formulas for Tp are different. The consistency condition between both formulas is 

Fa=^. (3.70) 
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This relation represents Newton's Law of Gravitation if Gs is written 0^= G M. 
Here, G refers to the gravitational constant, and M refers to the mass of the Sun. 
Therefore, Kepler's Third Law can be used for the derivation of Newton's Law of 
Gravitation, and vice versa. Measurements of G, see Gillies (1997), revealed that 
G = 6.6743 X 10 " mV (kg s'). By using Gs = 1.3291 x 10'° m' / s' derived from 
experimental data in Sect. 1.2.2 we find the correct value M = 1.99 x 10'" kg for 
the mass of the Sun. 



3.3.3 Stokes' Law 

Damping Force. Damping is relevant to many processes. Examples are given 
by Brownian motion, which is the motion of fine particles (pollen grains) that are 
suspended in water, the motion of a pendulum in air, and the motion of molecules 
in a fluid. In all these cases, the motion considered is damped by the presence of 
the surrounding particles of the medium, this means the surrounding particles act 
to reduce the velocity of objects considered. A simple formula for the damping 
force Fj, which is also called frictional force or drag force, is given by 

F,=-m-, (3.71) 

T 

see, for example, the Brownian motion model discussed in Chap. 6. Here, m is the 
mass of the (Brownian) particle considered, v is the particle velocity, and r is a 
characteristic time scale for the damping of particle motion. The basic idea of this 
assumption is that the damping force Fj is linear in the velocity. The negative sign 
appears because the damping force does always act to reduce the particle velocity. 
However, the assumption (3.71) does not specify the time scale v. The effect of 
the medium considered on this time scale is not explained. 

Dimensional Analysis. Let us calculate ron the basis of dimensional analysis. 
This time scale will depend on the dynamic viscosity /j that reflects the influence 
of the medium on the damping: the higher the viscosity, the smaller will be the 
damping time. The damping time will also depend on the particle mass m: the 
higher the mass, the longer it will take to damp out the motion. The damping time 
will also depend on the size of the particle considered: the bigger the particle, the 
smaller will be the damping time. For simplicity, we consider a spherical particle 
that is characterized by its particle radius r. Hence, we have to expect r as a 
function t= Tip, m, r). To use dimensional analysis we consider the constraint 

t' /j'' m' r'' = Ci, (3.72) 

where Ci is any constant. According to the dimensions of t, jU, m, and r (see 
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Table 3.1), this condition implies 

r|— ] M' L' = L"-" M"*'!"-'' = c^, (3.73) 

where Cj is any constant. The left-hand side is independent of L, M, and T if the 
following three conditions are satisfied, 

f) = d-b, (3.74a) 

= Z> + c, (3.74b) 

= a-b. (3.74c) 

We use the exponent a as an independent variable. The latter three conditions 
provide then the constraints 

d=b = a, (3.75a) 

c = -b = -a, (3.75b) 

b = a, (3.75c) 

where the last expression was used to replace b in the other two relations. Hence, 
Eq. (3.72) can be written 

r>°m-V = (^^l =c,. (3.76) 

y m ) 

Thus, by introducing the unknown damping constant c^ we find for r the relation 

m ,^ „„^ 

T = c,—. (3.77) 

Comparison with Other Observations. The constant Cj can be calculated by 
the consideration of consequences of the Navier-Stokes equations (which will be 
presented in Chap. 10). This analysis was performed by Stokes (1851). He found 
the expression F^^-dnj^rv for the damping force regarding the motion of 
spherical objects in fluids with a very small Reynolds number (for laminar flows). 
The result F^ = -6 K/jrv confirms the assumption (3.71) that the damping force is 
a linear function of velocity. In addition, Stokes' result provides the damping time 
scale: the combination of r= -mv/Fj with Fj = -6 TTjjrv implies 



671 jur 



(3.78) 



Hence, the damping constant is found to be c^ = 1/(6 ;r). Stokes' theoretical result 
is supported by measurements (Millikan 1910, 1917, 1923). The extension of 
Stokes' result to flows that are not characterized by a very small Reynolds number 
is provided by the Stokes-Cunningham formula (Li & Wang 2003). 
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3,4 Applications of Medium-Complexity: Time Measurement 

Let us consider now another example for the use of dimensional analysis: the 
measurement of time by a pendulum. This problem has a higher complexity than 
the problems discussed in Sect. 3.3: we will find a nondimensional equation that 
involves three nondimensional products. There is then the question of how is it 
possible to make the result of dimensional analysis useful, this means how is it 
possible to obtain an explicit formula for the definition of time on this basis. 



3.4.1 The Pendulum Period 

Time Measurements. The measurement of time is clearly relevant to our life: 
the specification of time is helpful to make appointments, time is used to analyze 
many processes (like weather or climate), and time is used to define other relevant 
variables (like velocity). Thus there is a long history of the measurement of time: 
sundials, water, candle, sand, and mechanical clocks were applied over centuries. 
Inspired by investigations of pendulums by Galileo Galilei around 1602, the pen- 
dulum clock was invented and patented by the Dutch scientist Christiaan Huygens 
in 1656. Galileo discovered the key property that makes pendulums useful 
timekeepers: isochronism, which means that the period of swing of a pendulum is 
approximately the same for differently sized swings. Huygens' pendulum clock 
was more accurate than any other measurement methods used before: his clock 
had an error of less than one minute per day. His later refinements reduced the 
clock's errors to less than 10 seconds per day. From its invention by Huygens until 
the 1930s, the pendulum clock was the World's most accurate timekeeper. During 
the Industrial Revolution (1700-1900), daily life was organized around the home 
pendulum clock. More accurate pendulum clocks, called regulators, were installed 
in places of business and used to schedule work and set other clocks. The most 
accurate pendulum clocks, known as astronomical regulators, were used in 
observatories for astronomy, surveying, and for celestial navigation. Beginning in 
the 1800s, astronomical regulators in naval observatories served as primary 
standards for national time distribution services. 

Pendulum. An illustration of a pendulum is given in Fig. 3.3. A mass m is 
attached to one end of a rigid, but weightless, supported rod of length r. The rod is 
free to rotate in one plane. The angle «o refers to the initial angle of displacement 
from the vertical. The pendulum is driven by the gravity force that is accounted 
for in Fig. 3.3 by the gravity acceleration g. Another force that may affect the 
pendulum motion is given by the damping force due to the air resistance. The 
damping force reduces gradually the amplitude of oscillations (see Fig. 3.3b). 
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a) Undamped Pendulum 



b) Damped Pendulum 



m ! 




■■}■ ' mg sma,, 
I 



mg cos a,, 




Gravity force mg 

Fig. 3.3. A pendulum. The left-hand-side figure illustrates an undamped pendu- 
lum that swings between two maximum positions at the same angle. The right- 
hand side figure illustrates a damped pendulum where the damping reduces the 
amplitude of oscillations. 

The damping force is caused by the fluid viscosity //. In contrast to the gravity 
force there is no exact knowledge available about the structure of the damping 
force. First of all, we would like to know how this force depends on the velocity 
(whether it is, e.g., a linear or quadratic function). Unfortunately, this dependence 
of the damping force on the velocity is often unknown. Usually, the effect of the 
damping force is involved by modeling this force according to Stokes' Law 
(3.71): see Sect. 9.4. 



3.4.2 Dimensional Analysis 



Pendulum Period. The most important variable of the pendulum problem is its 
period Tp, which is the time required for the pendulum bob to swing through one 
complete cycle and return to its original position. The relevance of Tp is given by 
the fact that Tp determines the unit for the measurement of time. To calculate Tp, 
we have to know which variables may affect Tp. According to the discussion in 
Sect. 3.4.1 we expect that Tp depends on the pendulum mass m, length r, and 
initial angle of displacement a^. Tp may also depend on the gravity acceleration g 
and viscosity //. Thus, we expect Tp as a function of the following parameters. 



Tp =Tp{aQ,r,g,m,iu). 



(3.79) 
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It is worth noting that the damped pendulum motion is not periodic because of the 
amplitude reduction. Nevertheless, the pendulum period Tp still can be used to 
characterize the pendulum motion. Tp is defined for this case as the time required 
for the mass to swing once forth and back (the time between t^, and ?2 in Fig. 3.3). 

Dimensional Analysis. According to Eq. (3.79), the requirement for a dimen- 
sionally correct relation for the pendulum period Tp is given by 



Tp «o r g m ju' =c,. 



(3.80) 



where Ci is any constant. By taking the dimensions of Tp, a^, r, g, m, and // into 
account, Eq. (3.80) reads 



r(L'M'ry L^\\\ M 
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(3.81) 



where C2 is any constant. The condition that the left-hand side is independent of Z,, 
M, and T then implies the following three conditions. 



= c + fi? - /, 

= e + f, 

= a-2d-f. 



(3.82a) 
(3.82b) 
(3.82c) 



We use the exponent a as an independent variable because we like to calculate Tp. 
In addition to a, we use /as an independent variable to keep the damping effect 
separated from the other variables (this means we would like to have a formula for 
the pendulum period where // appears only ones). Equations (3.82) provide then 
for c, d, and e the constraints 



a - f ^ 3f -a 
— Z^ + f = ^ — ' 






(3.83a) 
(3.83b) 
(3.83c) 



Here, Eq. (3.83c) was used to obtain Eq. (3.83a). Accordingly, Eq. (3.80) can be 
written 
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(3.84) 



Consequently, there are three nondimensional products that can be related to each 
other: Tp/ {r/ gY'^, a^, and jU {r^ / gY'^ / m. Hence, the pendulum period Tp can be 
written 



Tp =2ncp 



(3.85) 
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Here, a factor In Cp is introduced. The factor 2 n accounts for the usual relation 
between the period Tp and the related angular frequency (Op = 2 nl Tp. The non- 
dimensional product Cp in Eq. (3.85) can be a function of «o and // {P I gf'^ I m, 

Cp=Cp{a„,jU,). (3.86) 

The nondimensional variable jU, = jU {r' I gf^^ I m is introduced here as an abbre- 
viation to simplify the notation. 



3.4.3 Comparisons with Other Results 

Nonlinear Pendulum Equation. To use the pendulum period relation (3.85) 
for calculations of Tp we need to know how the coefficient Cp depends on «(, and 
the damping variable //,. One way to address this problem would be the attempt to 
use measurements for the investigation of Cp = C/>(«o, f^*)- This approach is rather 
expensive, and it is affected by randomness. A much more appropriate approach is 
given by the use of the pendulum equation for the calculation of Cp. The latter 
equation is a consequence of Newton's Second Law. By defining the gravity and 
damping forces, the equation for the pendulum written in terms of nondimen- 
sional parameters reads (see Sect. 9.4), 

d^a da . 

— Y = -d/it, sino;. (3.87) 

dt, dt. 

Here, a = df) is the pendulum angle that changes in time, and t, = t/ (r/ gY'^ is a 
nondimensional time. The two contributions on the right-hand side of Eq. (3.87) 
account for the effect of gravity (the last term) and damping (the term involving 
//.). The factor d refers to a nondimensional parameter that depends, for example, 
on the fluid considered and the properties of the pendulum bob. By adopting 
Stokes' Law of Friction, it turns out that d ^ 6 nrpi r where Vp refers to the radius 
of the pendulum bob. However, the validity of Stokes' Law is questionable in 
many cases, such that d is considered here simply as any unknown parameter. The 
solution of equation (3.87) requires the specification of initial values. These initial 
values are given by ait = 0) = «o ^"d dal du{t = 0) = 0. Unfortunately, Eq. (3.87) 
cannot be solved analytically because of the nonlinearity (the sine function) 
involved. 

Linear Pendulum Equation and its Solution. However, for a relatively small 
initial angle of displacement a^ it is possible to solve Eq. (3.87). For this case, Eq. 
(3.87) reads 

fi?'« da 

— :r = -dLL, a, (3.88) 

dU^ dt, ' 
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because sin a can be approximated by a. As shown in Sect. 9.4, the solution of 
Eq. (3.88) combined with the initial conditions considered is given by 
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(3.89) 



if the condition d ju, < 2 is satisfied. The latter condition simply means that the 
damping should be not too strong. In Eq. (3.89), Sis defined by 
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and Tp'''^ is given by the expression 
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The superscript "{LIN)" here refers to the value of Tp provided by the solution 
(3.89) of the linear equation. According to its definition Cp = Tp/[2 ;r(r/g)"^], the 
coefficient Cp is then given by 
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(3.92) 



It is worth noting that the solution (3.89) of the linear pendulum equation (3.88) 
implies that Cp^''"^* is independent of the initial angle of displacement a^. To prepare 
the discussion below we also use the solution (3.89) of the linear equation for the 
calculation of the ratio 
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(3.93) 



Here, a-i^"^' is the angle at the time ?2 (see Fig. 3.3) calculated by the solution of 
the linear pendulum equation. The ratio a-j;^'^' I Oq describes the reduction of the 
maximum angle due to damping. The linear pendulum equation implies that the 
ratio 02''"^' / Go is found to be independent of the initial angle of displacement a^. 

Range of Damping Considered. To see the range of applicability of the linear 
Eq. (3.88) and its consequences we have to solve numerically the nonlinear 
Eq. (3.87). This requires an appropriate choice of the d ju, range considered. The 
latter question is addressed in terms of Fig. 3.4a that shows the angle reduction 
due to damping after the first two swings for djU, variations 0.01 <d /j,< 1. It may 
be seen that the consequence a-l^"^' I a^ of the linear pendulum equation, which is 
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Fig. 3.4. The nonlinear damped pendulum, a^ I «o is shown in (a) in dependence 
on the initial angle of displacement a^. Here, «2 is the angle at the time U (see 
Fig. 3.3). The values on the curves refer to the d ju, values applied, (b) shows Cp 
for the nonlinear pendulum in dependence on the initial angle of displacement «o. 
The d^, values applied are given by 0, 0.01, 0.05, 0.1, and 0.2, respectively. 



independent of a^, is applicable for relatively small «o values. We also observe 
that relatively small damping values d /^i, already result in a significant maximum 
angle reducfion. For example, for d /lu = 0.2 we find a^l a^^ 0.532 for very small 
«o values. Thus, we will only consider a range d lu, < 0.2 in the following to allow 
the pendulum at least a few swings. For higher damping values it turns out that the 
concept to calculate the pendulum period Tp for the damped pendulum becomes 
questionable: it is hardly possible to talk about a pendulum period if the pendulum 
motion disappears after a few swings. 

Nonlinear Damped Pendulum Period. The pendulum period of the nonlinear 
damped pendulum can be calculated by solving the nonlinear pendulum equation 
(3.87) numerically. The results obtained for the range < djj, < 0.2 of dju, values 
are shown in Fig. 3.4b. For relatively small «o the Cp values correspond to the result 
(3.92) of the linear pendulum equation, which is independent of a^. All the Cp'^"^ 
values are found to be very close to 1 because of the range ofd/u, considered. For 
example, we have c/^'^' = 1.005 for the highest damping value d /^i, = 0.2 applied. 
The Cp value for the undamped (d /^i, = 0) nonlinear pendulum shows a significant 
increase of Cp values for initial angles Gq - 90°. The reason for this increase is the 
following. The sin a function is always smaller than a. Thus, the gravity force m g 
sin a that drives the pendulum motion is smaller for the nonlinear case than for the 
linear approximation. The smaller driving force of the pendulum is then related to 
a longer time required to swing once forth and back, i.e., a higher Cp and Tp. The 
effect of damping on Cp for the range «o - 90° is such that the Cp values are smaller 
than for the undamped pendulum. An explanation for this finding is given by the 
reduction of the maximum angle values due to damping. The smaller angle varia- 
tion reduces the time required for the first two swings of the pendulum. 
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Table 3.2 The Cp values of the damped nonhnear pendulum in dependence on the initial angle ffj 
and damping value c///,. 

djd, ao=5° «o=10° ao=15° ao=20° a^^2S° «o=30° 



0.00 


1.0005 


1.0019 


1.0043 


1.0077 


1.0120 


1.0174 


0.01 


1.0005 


1.0019 


1.0042 


1.0074 


1.0117 


1.0169 


0.05 


1.0007 


1.0020 


1.0040 


1.0069 


1.0106 


1.0152 


0.10 


1.0016 


1.0027 


1.0045 


1.0070 


1.0103 


1.0143 


0.20 


1.0053 


1.0062 


1.0077 


1.0097 


1.0123 


1.0155 



Seconds Pendulum. A look at the Cp values shown in Table 3.2 for various 
initial angles of displacement and damping values shows that Cp= \ represents a 
very good approximation if the damping and initial angle of displacement are not 
too high. For the damping values applied one finds that the deviations of Cp values 
from 1 are smaller than 1% if «o< 20°. Thus, the pendulum period is given by 

Tp =ln\- (3.94) 

for «o < 20°. This formula reveals the requirements for the design of a seconds 
pendulum, which was used for the definition of time after 1670. A seconds pendu- 
lum is a pendulum whose period is precisely two seconds, one second for a swing 
forward and one second for a swing backward. By setting Tp = 2 ■& and using g = 
9.81 m/ s" for the gravity acceleration, we find the requirement r = 0.9940 m, this 
means r has to be about 1 m. ft is worth noting that the seconds pendulum can be 
also applied for the definition of length. In 1790, Talleyrand proposed that the 
meter is the length of the seconds pendulum at a latitude of 45°. 



3,5 Applications of High-Complexity: Lift 

After demonstrating the value of dimensional analysis in Sects. 3.3 and 3.4, let 
us come back to the lift problem considered in the beginning. This problem has a 
higher complexity than the problems discussed before: it involves a nondimen- 
sional equation that relates five nondimensional products. The focus here is not on 
the use of dimensional analysis (the nondimensional equation for this problem 
was derived in Sect. 3.2), but on the following. The nondimensional equation 
provided by dimensional analysis is not really helpful without having a method to 
close this equation by the determination of the lift coefficient: the lift coefficient 
has to be known to calculate the lift force. The determination of coefficients can 
be a hard task if the coefficient is a fimction of many parameters (as given for the 
lift coefficient that is a function of four parameters). This determination usually 
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requires solutions for four problems: (1) the understanding of the meaning of 
parameters involved, (2) the imderstanding of the range of parameter variations 
that have be considered, (3) the finding of a basic solution (the determination of 
the influence of the process-controlling parameter), and (4) insight into modifica- 
tions of the influence of the process-controlling parameter due to variations of 
other parameters. This process will be illustrated here regarding the relevant lift 
force problem (a 1% increase in maximum lift coefficient is equivalent to a 
2000 kg increase in payload at a fixed approach speed). 



3.5.1 The Problem 

Lift Coefficient Formula. To prepare the following discussion let us have a 
closer look at the meaning of parameters involved in the lift force calculation. 
According to Eq. (3.20), the lift force 7^^ = C^ p v^ dj I 2 depends on the lift 
coefficient Q that is given by 



Q=Q 



f 1 \ 

' Ma, Re 



d-, 



(3.95) 



Here, the Mach number Ma is defined by 

Ma = -, (3.96) 

s 

and the Reynolds number Re is defined by 

Re=P^ = ^. (3.97) 

The last expression results from the definition of the kinematic viscosity v= /ul p. 
Equation (3.97) represents one way to introduce the Reynolds number. Another 
possibility would be given by the use of di instead of Jj in Eq. (3.97). In Eq. 
(3.95), the lift coefficient is written as a function of Re instead of a function of 
Re\ This writing does not make any difference because the dependence of Q on 
Re is unknown anyway. 

Mach Number. What is the meaning of the Mach number? The Mach number 
Ma measures the compressibility of a fluid. When you pump up a tire you can fill 
it with air, and then pump in more air, with no appreciable change in volume. You 
cannot do the same with water. Thus, air is compressible (it has a relatively high 
compressibility) and water is incompressible (it has a relatively low compressibil- 
ity). The high compressibility of air can affect the motion of objects in air. When 
you are driving along in your car at 120 km/h, air seems to be incompressible. 
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because when you push at such a speed, the air has plenty of time to get out of the 
way. At these relatively low speeds, the air behaves just like water. However, as 
your aircraft speeds up, and Ma « 0.3, the air molecules cannot get out of the way 
fast enough, so there are the first signs of changes in density in the air around the 
aircraft. Obviously, for higher Mach numbers there is a much stronger increase in 
density in the air around the aircraft. 

Reynolds Number. What is the meaning of the Reynolds number? The inverse 
Reynolds number 7?e ' represents a measure for the viscosity of a fluid. Viscosity 
is a measure of the resistance of a fluid to be deformed or to flow. Fluids may 
have a very different viscosity: the dynamic viscosity of air, for example, is much 
smaller than the dynamic viscosity of water. The reason for that is given by the 
different intermolecular forces of air and water: the intermolecular forces (which 
hinder the flow) of water are much stronger than the intermolecular forces of air. 
The viscosity may have a relevant influence on the motion of objects in fluids. For 
the same propulsion power, the speed of the object will be higher in a fluid with a 
lower viscosity. 



3.5.2 The Range of Parameter Variations 

Angle of Attack Variation. A reasonable range of the 9 variation can be easily 
determined: the angle of attack should be in the range Q< 9< 45°. 

Shape Factor Variation. Figure 3.5 illustrates the possible range of variations 
of the shape factor J, /fi?2- For the NACA 0010 and NACA 0020 airfoils, the ratios 
of maximum width to maximum length are given by 0.1 and 0.2, respectively. 
These ratios can be used to characterize the ratio di I t/j. Thus, a reasonable range 
ofdJd2 variations is given by 0.1 <dild2< 0.2. 

Mach Number Variation. To determine the range of variations of the Mach 
number Ma ^ v / s we need to specify the variation of the velocity of sound s and 
the relative aircraft velocity v. For dry air at standard atmospheric pressure, s is 
given by Eq. (3.40). To cover a range of sound velocity variations we considers at 
20°C (where s = 343.4 m/s = 1236 km/h) and s at -40° (where s = 306.3 m/s = 
1103 km/h). The corresponding curves for the Mach number are shown in Fig. 
3.6 for velocities up to 2000 km/h. It may be seen that the range of Mach number 
variations is < Ma < 1.8. 



^ 



Fig. 3.5. NACA 0010 and 0020 airfoils {solid and dashed lines, respectively). 
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Fig. 3.6. The dependence of the Mach number Ma and Reynolds number Re on 
the velocity (in km/h). The solid and dashed lines refer to sound velocity and 
viscosity values that are calculated for a temperature of 20°C and -40°C, respec- 
tively. Here, (ij ^ 3 m is assumed. 

Reynolds Number Variation. The Reynolds number Re = d2V I v depends on 
the viscosity vand the relative aircraft velocity v. For air at standard atmospheric 
pressure, the viscosity is given by Eq. (3.39). To have reference values, we will 
consider the kinematic viscosity at 20°C (where v = 1.55 x 10"^ m^ / s) and at 
-40°C (where v= 1.04 x 10"^^ m^/ s). The corresponding curves for the Reynolds 
number are shown in Fig. 3.6 for velocities up to 2000 km/h, where (i2 ^ 3 m is 
used. It may be seen that the Reynolds number may vary over many orders of 
magnitude. In particular, we have to expect values \(f < Re < 10'. The temper- 
ature has an influence on the Reynolds number, but the effect is rather limited in 
comparison to the huge variations of the Reynolds number with the velocity v. 



3.5.3 The Lift Coefficient Calculation 



Lift Coefficient Calculation. The lift coefficient can be calculated by means of 
the computational fluid dynamics (CFD) code XFOIL (http://web.mit.edu/drela 
/Public/web/xfoil/). It is worth emphasizing that this calculation does not exactly 
provide the lift coefficient Q as defined by F^^ Ci p v^ d} I 2. Usually, Q is 
calculated by using the wing area instead of d2' in 7^^ = Q /? v^ d2 I 2. The wing 
area is determined by the chord and the span length. The span length does not 
enter Fi= Cipv^ d^ II. However, one may expect that the span length is compa- 
rable to (^2- Depending on the shape of the wing, the lift coefficient calculated by 
the CFD code is then equal to /? Q, where the constant P is of order unity. To 
focus on the main features, we will assume here for simplicity that P=\. 
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Fig. 3.7. The lift coefficient Q (solid curve) of a 
NACA 0010 airfoil for Re = 10' and Ma = 0. The 
stall angle indicated by the dashed line is 19.6°. 
The linear curve Q = 2 ;r ^ ^ / 1 80 shows the pre- 
diction of the thin airfoil theory. 



Angle of Attack Effects. Let us understand first the basic behavior of the lift 
coefficient that is determined by its variation with the angle of attack 9 (which is 
the most relevant variable). A typical lift coefficient curve obtained with the CFD 
code is given in Fig. 3.7. For relatively small angles we find a very good 
agreement of Q with the linear function Ci = 2 n^ 01 180 that is provided by the 
thin airfoil theory (Abbot & Doenhoff 1959). This linear increase is the expected 
behavior: more air passing under the wing is deflected downward by the bottom 
surface of the wing. Thus, the pressure below the airfoil increases whereas the 
pressure above the airfoil decreases. The vertical pressure difference results in the 
lift force that pushes the airplane up. But there is a limit to this, i.e., there is a 
maximum lift coefficient at the stall angle. For angles higher than the stall angle 
there are no well-organized high and low pressure areas anymore: the pressure 
distribution now becomes random due to the appearance of turbulence (the high- 
pressure air under the wing can more easily move around the back of the wing 
toward the low-pressure air on top of the wing, thus weakening that low-pressure 
area). Thus, the lift coefficient will decrease for angles higher than the stall angle. 

Mach Number Effects. The effect of Mach number variations on the lift coef- 
ficient is shown in Fig. 3.8a regarding the NACA 0010 airfoil. An increasing 
Mach number reduces the lift coefficient significantly. This observation can be 
explained by the increasing air density around the aircraft: the air molecules 
cannot get out of the way fast enough. The increasing air density then decreases 
the vertical pressure difference that pushes the airplane up. We see a strong 
reduction of the stall angle, which has implications on the flight behavior. 

Reynolds Number Effects. The effect of Reynolds number variations on the 
lift coefficient is shown in Fig. 3.8b for the NACA 0010 airfoil. A decrease of the 
Reynolds number Re = d2V I v can be seen as a reduction of the relative aircraft 
velocity v. The latter implies a reduction of the amount of air that is deflected 
downward, which decreases the lift. The decrease of the lift coefficient for this 
case looks similar to the effect of an increasing Mach number. A difference is 
given by the fact that the lift coefficient still follows the prediction of the thin 
airfoil theory for relatively small angles of attack. 
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Fig. 3.8. Lift coefficients for the NACA 0010 airfoil {upper pictures: d^^ ^ 0.1) 
and NACA 0020 airfoil {lower pictures: d^2 = 0.2). In (a) and (c), the Reynolds 
number Re= 10' is fixed and Mach number variations are considered (the values 
related to the curves show Ma). In (b) and (d), the Mach number Ma = is fixed 
and Reynolds number variations are considered (the values related to the curves 
show Re). The dashed lines show the thin airfoil curve €^ = 2 n^ 91 180. 

Shape Factor Effects. The effect of fi?i I d2 variations is given by Figs. 3.8c-d 
that show the effect of Mach and Reynolds number variations for the NACA 0020 
airfoil. The trend of these curves is similar to the trends given in the upper pictures 
for the NACA 0010 airfoil. However, dil dj does affect these trends: we see that a 
higher 4/4 slightly increases the deviations from the thin airfoil curve Q = 

2 71^ eiim. 



3,6 Summary 



This chapter was organized into two parts. First, the use of dimensional analy- 
sis was explained in Sect. 3.2. Second, applications of such analytical results to 
problems of increasing complexity were discussed in Sects. 3.3-3.5 by combining 
the conclusions of dimensional analysis with the results of simulations, theory and 
experiments. Let us summarize the basic observations made here. 
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Dimensional Analysis. Dimensional analysis allows us to look at problems in 
their simplest form. Regarding the lift force, for example, we see that the lift force 
Fj^ = Fi(v, di, dj, 0, p, n, s), which is a function of seven variables, cannot be any 
function, but only certain function types are dimensionally correct equations. Such 
a correct equation is F^ = Cipv^ d-^ 12, where Q is a function of four parameters, 
Ci = Ci{9, dt I d2, Ma, Re). The function obtained by dimensional analysis is not 
unique: another correct equation would be Fi= C^pv^ d{' 1 2. The latter equation 
can be also written 7*'^ = (C^ d^ I d2) p v^ d2 1 2, this means the difference between 
both equations is a modified lift coefficient. The conclusions obtained by dimen- 
sional analysis are very helpful: 

• First, we can use the result of dimensional analysis to make relatively reasonable 
predictions without having any additional information. Coefficients like Q here 
are often found to be of order unity. For example, a look at Fig. 3.8 shows that 
usual Ci variations are between zero and two. Thus, by using a value Q = 1 we 
are already in the position to use the lift force formula for obtaining reasonable 
estimates. 

• Second, the use of dimensional analysis is helpful for the understanding of the 
meaning and the relevance of parameters like the Mach and Reynolds numbers. 
We can understand the effects represented by these numbers (see the discussion 
in Sect. 3.5.1), and we can assess the conditions under which such effects can be 
disregarded. For example, the effect of gravity on the lift coefficient can be 
neglected because there are usually conditions such that (t/j gf'^ « ^ (see the 
discussion in Sect. 3.1). With the same sort of reasoning we can conclude that 
the lift coefficient is independent of the Mach number if the relative aircraft 
velocity v « s. Similarly, we may expect that a high Reynolds number Re> 10* 
(which is often given in reality) will have little effect on the lift coefficient for a 
reasonable range < < 20° of angle of attack variations. The latter view is 
supported by Figs. 3.8b-d. Such scale analysis represents a powerful tool to 
simplify equations (in addition to the problem reduction that is immediately a 
consequence of the use of dimensionally correct equations). 

• Third, another relevant advantage of dimensional analysis is the possibility to 
design accurate experiments even if the real system (aircraft, ship, submarine) 
cannot be investigated. The key for doing this is the design of a similar model 
system. This concept does not only mean that the real and the model system 
have to be geometrically similar. The use of this concept requires the equality of 
all nondimensional parameters involved. Under conditions where a complete 
similarity cannot be established it is possible to design relatively similar model 
systems that enable the study of the most relevant system features. An example 
for the design of a relatively similar system was given at the end of Sect. 3.2.3, 
where the Mach number constraint was disregarded. 
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Simulations and Related Theory. Dimensional analysis does never provide 
complete solutions for a problem: there are always unknown constants that have to 
be specified. The determination of such constants requires the combination of 
conclusions of dimensional analysis with the results of simulations (the numerical 
solution of nonlinear differential equations that cannot be solved analytically), 
theory (analytical consequences of differential equations) and experiments. The 
most efficient approach to completely solve problems that are characterized by 
nondimensional equations with more than one nondimensional product is the 
combined use of simulations and theory. Why do we need both simulations and 
theory? The advantage of simulations is that such results can be obtained for more 
complex cases than the simple cases for which analytical results are available. The 
disadvantage of simulations is that such results are often only available in tenns of 
plots, and the correctness of such results has to be justified. Such justification for 
simulation results can be provided by the agreement with analytical results, and 
analytical results are much easier to use and to understand. Thus, the combination 
of both methods is the most convenient way for problem solutions. The success of 
such a combined use of simulations and theory was shown by means of the dis- 
cussion of the pendulum and lift problems in Sects. 3.4 and 3.5, respectively. The 
simulation results were given by solutions of the nonlinear damped pendulum 
equation and the lift coefficient calculation in terms of the CFD code (which gives 
an approximate solution to the Navier-Stokes equations). The simulation results 
were justified by their agreement with analytical conclusions of the differential 
equations considered (the solution of the linear damped pendulum equation and 
the thin airfoil curve Q = 2 ;r^ ^/ 180, respectively). 

Experiments. Experiments usually cover only a limited range of the conditions 
that can be covered by simulations, but they have an essential advantage: many 
experiments can provide information that is independent of assumptions (every 
assumption can be correct or not). This characteristic property of experiments is 
very valuable: the comparison of the results of simulations and theory with exper- 
iments represents the way to confirm the assumptions that are represented in terms 
of equations. For example, the pendulum period formula Tp^ 2 ;r(r/g^)"^ is 
implied by assumptions regarding the interaction of the pendulum acceleration, 
gravity and damping forces. In principle, such ideas about the structure and inter- 
action of forces can be wrong. Thus, experiments (regarding the variation of Tp 
with r, the influence of the initial angle of displacement and the influence of the 
mass of the pendulum bob) are needed to confirm the correctness of theoretical 
results. For simplicity, the use of experimental results was illustrated here only for 
relatively simple problems that do only involve one nondimensional product. The 
discussion of the vehicular stopping distance, Kepler's Third Law, and Stokes' 
Law in Sect. 3.3 revealed the advantage of experiments. 
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3.7 Exercises 



3.2.1 Assume that the hft force F^ is determined by the relative aircraft velocity 
V and the total surface area A of the tops of the wings. 

a) Which additional parameter will affect 7*'^ due to dimensional reasons? 

b) Use the dimension of Fj^ and the parameters involved to find directly a 
relation for F^ that is dimensionally correct. 

3.2.2 The position sofa moving body at time t is described by the function 
s - a + bt + ct^. 

a) Determine the dimension of the parameters a, b, and c. 

b) Explain how a, b, and c are related to physical properties of the problem 
considered. 

3.2.3 One model for the damping force Fj assumes that Fj is a linear function of 
the velocity u of a moving body, Fj= -a v. Another model for Fj assumes 
that the damping force is a quadratic function of u, Fj = -b v'. 

a) Determine the dimension of the parameters a and b. 

b) Explain how a and b can be related to properties of the moving body. 

3.2.4 Assume that the damping time r of the motion of a particle in a fluid is a 
function of the particle radius r and the kinematic viscosity v of the fluid. 

a) Use the dimensions of r, r, and v to find directly a relation for r that is 
dimensionally correct. 

b) Does this relation represent a reasonable relation for r? 

3.2.5 Consider a raindrop falling from a cloud. Assume that the velocity v of the 
raindrop depends on the gravity acceleration g. 

a) Find one combination of v and g that has the dimension T. Find another 
combination of v and g that has the dimension L. 

b) To find a nondimensional relation between v, g, and another variable we 
have to involve a variable that has the dimension T or L. Explain which 
additional variable (with dimension L or T) should be involved. 

c) Combine v, g, and the additional variable in a nondimensional product. 

3.2.6 Consider the following equation for a spring-mass system (see Sect. 7.3), 

d^y dy , _ 
m — Y + 7 +i^y = 0- 
dt dt 

Here, y{t) is the displacement (dimension L) of the mass from its equilibri- 
um position at time t. The constants involved are the mass m, the damping 
constant y, and the spring constant k. 
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a) Determine the dimension of ;Kand k . 

b) Use m, k, and t to define a nondimensional time t,. 

c) Rewrite the equation such that only the nondimensional time t, and 
nondimensional parameters appear in addition to y. 

3.2.7 Consider the equation P, = J{P2, Pi, A) for ^ny nondimensional products 
P„P2, Pi, and P,. 

a) What is the problem regarding the use of this equation? 

b) How is it possible to overcome this problem? 

c) Given the case that the real system cannot be studied directly: what are 
the conditions for the design of a completely similar model system? 

d) Given the case that a complete similarity cannot be achieved: how is it 
possible to design a relatively similar model system? 

3.2.8 Consider the lift force problem discussed in Sect. 3.2. The list of nondi- 
mensional products obtained is given by 



^L n '^i -" 



0, 



pv d^ d^ pd^v V 

a) Use this list to identify the independent and dependent variables applied. 

b) Assume that the gravity acceleration g has to be involved in the set of 
variables considered. Which dependent variables used in the given list 
can be applied for finding a nondimensional product that involves g^? 
Find the nondimensional product that involves g. 

c) The result of nondimensional analysis can be written F[= Cipv'^ d2 1 2. 
Generalize the function Q that follows from the list above by involving 
the nondimensional product that involves g. 

3.2.9 Consider the function F^ ^ C^pv^ d^ II, where Q is given by the result of 
exercise 3.2.8. 

a) What are the conditions for a completely similar model system? Account 
for the fact that g cannot be changed. 

b) Assume that d-^ = a d^, where < « < 1 . Specify for each variable the 
ratio between the variable in the model system and the variable in the 
real system (e.g., d^^ I d^ = ■■■, v'^ Iv" = ...) as a function of «. 

c) Specify the latter design conditions for the case that the same fluid is 
used in the real system and in the model system. 

3.2.10 The period of a damped pendulum is given by Tp^ Incp {rlgf'^, where the 
factor is given by the function Cp = Cp{a^, /u,). The damping parameter is 
defined by //. = // {P I g)"^ / m. Here, r is the length of the pendulum, g is 
the gravity acceleration, «o denotes the initial angle of displacement, // is 
the dynamic viscosity, and m is the pendulum mass. 
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a) Describe an experiment for the calculation of Cp. 

b) Derive the design conditions for a model pendulum that is 50% smaller 
than a real pendulum. 

c) Specify the design conditions for the case that the same fluid is used. 

3.3.1 What is the power P (dimension M L^ T^) that is required to keep a vehicle 
of length d and mass m moving at a constant speed v7 

a) Use dimensional analysis to calculate the power P. 

b) Rewrite your result in terms of the kinetic energy K = m v^ 1 2, and the 
characteristic time scale t= div. 

3.3.2 The hydrostatic pressure of blood in humans is a part of the total blood 
pressure. The hydrostatic pressure P (dimension M Z ' T') is considered to 
depend on the blood density p, the height h of the blood column between 
the heart and some lower point in the body, and the gravity acceleration g 
(Giordano et al. 2003). 

a) Use dimensional analysis to calculate the hydrostatic pressure P. 

b) Pressure is defined as force per area. Rewrite your result for P by taking 
reference to the gravity force Fg = m g and an area A that you have to 
define conveniently. 

3.3.3 For laminar flow in a pipe, the volume flow rate q (dimension V' 7^') is a 
function of the pipe radius r, the viscosity /u of the fluid, and the pressure 
drop per unit length dpi dz (dimension ML"^ T^). 

a) Use dimensional analysis to calculate the flow rate q. 

b) How does q change if the radius is increased by a factor of two? 

3.3.4 A star represents a liquid body that is held together by its own gravity. 
Stars may vibrate in several ways. The frequency (o of vibrations can be 
expected to depend on the radius r of the star, the star's density p, and the 
gravitational constant G (dimension Af ' L^ T^). 

a) Use dimensional analysis to calculate the frequency co of vibrations of 
stars. 

b) Explain the consequence of this result. 

3.3.5 The molecules of a gas interact in terms of attractive and repulsive forces. 
The attractive forces are considered to be negligible. The repulsive forces 
are considered to follow the relation F = K r". In this relation, r is the 
distance between the centers of two molecules, n is any exponent, and K is 
a constant of proportionality. The dynamic viscosity // of the gas depends 
on the thermal velocity u of a molecule, the mass m of a molecule, and the 
coefficient of repulsion iT. 
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a) Find the dimension ofK. 

b) Use dimensional analysis to calculate the dynamic viscosity // of the gas. 

c) The thermal velocity is related to the absolute temperature T (in °K) by 
the V = {kT I rnf'^, where k is Boltzmann's constant. Use v = {k Tl nif^^ 
to replace the y-dependence of the dynamic viscosity by a T-depen- 
dence. Determine the variation of the temperature dependence of the 
dynamic viscosity for variations 5 < « < oo. 

3.3.6 Consider Stokes' problem: a small sphere falling under gravity in a viscous 
fluid. The sphere is so small that the motion is very slow. The velocity v of 
sphere depends on the radius r of the sphere, the gravity force F^ and the 
dynamic viscosity // of the fluid. 

a) Use dimensional analysis to calculate the sphere's velocity v. 

b) Consider the case that the gravity force F^ and damping force F^ balance 
each other, this means we have Fg + Fj = 0. Assume that the damping 
force is given by Stokes' Law, Fj = -6 7i/urv. Use this fact to determine 
the unknown constant in the expression for ;; obtained by dimensional 
analysis. 

c) The density /?, = ml V^ of the sphere is given in terms of the mass m and 
volume F, = 4;rrV3 of the sphere. The gravity force is given by the 
relation F^^ mg ^ p^V, g, where g refers to the gravity acceleration. 
Find Stokes' result for v by using F^ = p^ V,g in the expression for v 
obtained above. 

3.4.1 Consider again Stokes' problem: a small sphere falling under gravity in a 
viscous fluid. The sphere's velocity v is assumed to depend on the sphere's 
radius r, the gravity acceleration g, the dynamic viscosity // of the fluid, 
and the sphere's density p^. 

a) Use dimensional analysis to identify two nondimensional numbers: the 
Reynolds number Re that is proportional to /j ', and the Froude number 
Fr that is proportional to g'\ 

b) Stokes' analysis showed that this problem can be described by only one 
nondimensional number: the ratio Fr I Re. Find the relation for v that is 
implied by this conclusion. 

3.4.2 Consider again Stokes' problem described in exercise 3.4.1. Consider the 
case that v depends in addition to r, g, //, and p^ on the fluid density p. 

a) Stokes' analysis showed that this problem can be described by two 
nondimensional numbers: the ratio Fr I Re and the ratio pi Ps- Find the 
relation for v that is implied by this conclusion. 

b) Extend the result obtained for exercise 3.4.1 by assuming that v does 
depend on p and p^ only via p^- p. 
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3.4.3 Consider a projectile that is fired with initial velocity Vq at an angle a with 
the horizon. 

a) Use dimensional analysis to calculate the projectile range r. You may 
expect that the gravity acceleration g affects r. 

b) You may expect that r depends on a via the function sin(« a), where n is 
a constant. Determine the constant n by considering special cases like 
« = 0, 71/4, nil. 

3.4.4 Consider this case: A windmill is rotated by air flow to produce power to 
pump water. You have to find the power output P (dimension ML^ T^) of 
the windmill. It can be expected that P depends on the diameter d of the 
windmill, the wind speed u, the air density p, the air viscosity //, and the 
rotational speed o (dimension 7^') of the windmill (Giordano et al. 2003). 

a) Use dimensional analysis to calculate P. Use the exponents of P, //, and 
o as independent parameters. 

b) Provide the design condition for a model that is q times smaller than the 
real windmill. Use the same fluid in the model system. 

3.4.5 Consider a steady laminar fluid flow through a smooth horizontal pipe. The 
pressure drop Ap (dimension ML ' 7"^) between two points along the pipe 
depends on the distance d between the two points, the diameter D of the 
pipe, the fluid density p, the fluid viscosity //, and the fluid velocity v. 

a) Use dimensional analysis to calculate Ap. Use the exponents of Ap, jU, 
and d as independent parameters. 

b) Provide the design condition for a model that is q times smaller than the 
real system. Use the same fluid in the model system. 

3.5.1 The lift force 7^^= C^pv' d2 l'2. is given in terms of the chord length ^2, the 
relative aircraft velocity v, and the air density p. The lift coefficient can be 
described by the thin airfoil formula Q = 2 k^ ^/ 180, where (9 is the angle 
of attack. 

a) Provide the design condition for a model that is q times smaller than the 
real system. 

b) Calculate for this case the ratio F^* I F^ of the lift force F^ in reality to 
the model lift force Fl\ The fluid and the relative aircraft velocity v are 
the same in reality and the model system. 

3.5.2 If a drop of liquid falls into a pool, a small column of liquid splashes out of 
the pool. The height h of the column depends on the mass m of the drop, 
the velocity v of the drop, the surface tension a of the liquid (dimension 
MT'^), the mass density p of the liquid, the viscosity /u of the hquid, and 
the gravity acceleration g (Langhaar 1951). 
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a) Use dimensional analysis to identify four nondimensional numbers: the 
length ratio H that is proportional to h, the surface tension T that is 
proportional to a, the Reynolds number Re that is proportional to // ', 
and the Froude number Fr that is proportional to g '. Use the exponents 
oih, /u, g, and eras independent parameters to find these numbers. Write 
these numbers by using the length scale h^ = (m/pY'^. 

b) Present a formula for the calculation ofh by using T, Fr, Re, and h^. 

3.5.3 An airplane is warming up its engine on the ground. The intensity of sound 
energy U (dimension ML ' F^) from the propeller at a distance d ahead of 
the airplane depends in addition to d on the diameter D of the propeller, the 
rotational speed o (dimension 7^') of the propeller, the mass density p of 
air, the air pressure/?, and the air viscosity // (Langhaar 1951). 

a) Use dimensional analysis to identify four nondimensional numbers: the 
nondimensional intensity of sound energy U, that is proportional to U, 
the nondimensional distance d, that is proportional to d, the nondimen- 
sional rotational speed o. that is proportional to o, and the nondimen- 
sional viscosity jU, that is proportional to //. Use the exponents of U, d, 
CO, and // as independent parameters to find these numbers. Write these 
numbers by using the velocity scale v^ = (pi fSf'^, which is proportional 
to the velocity of sound. 

b) Present a formula for the calculation of U by using d,, co,, ju,, and p. 
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The discussions in Chaps. 1 and 2 showed that randomness of data is relevant 
to actually all problems that involve observed data. For example, we have to deal 
with randomness to analyze the basic trend of noisy data of the global temperature 
increase. The way to address randomness effects in Chap. 2 was to minimize the 
deviations between a model and observations. First, there are questions about the 
validity of the approach applied: the use of the least-squares error was shown to be 
the most convenient choice among the three error concepts considered, but this 
approach of looking at the problem does not provide any sort of theoretical 
justification for the use of the least-squares error. Second, the problem considered 
(to find a model for the correlation of two variables) does only represent one par- 
ticular problem among many questions related to the development of models that 
involve randomness. In particular, we need mathematical concepts for the analysis 
of stochastic states to determine the mean value of variables like the atmospheric 
temperature, the typical amount of randomness, and the probability for finding 
special events (very high or low temperatures). On this basis we are interested in 
modeling concepts that we can use for the description of properties of random 
variables. We need such analysis and modeling concepts for several interacting 
random variables (like the three interacting components of the atmospheric wind 
vector). Apart from the analysis and modeling of stochastic states at a particular 
time, our predominant goal is the development of methods for the analysis and 
modeling of stochastic processes that evolve in time, which is a requirement for 
the analysis and prediction of many processes in nature and technology. These 
questions will be successively discussed in Chaps. 4, 6, 8, and 10. By taking 
reference to analysis and modeling concepts for the description of several random 
variables, it will be shown that the specific correlation problem addressed in 
Chap. 2 (the finding of an optimal model for observations) can be formulated and 
treated in a much more convenient way (see Chap. 10). 
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Fig. 4.1. An illustration of 100 measurements of 
the temperature T (in °C) in the stably stratified 
atmospheric boundary layer. Here, n refers to the 
number of the measurement value. 



In preparation of the discussions in Chaps. 6, 8, and 10, this chapter addresses 
the analysis and modeling of properties of the state of one random variable. The 
requirement for the development of such analysis and modeling concepts will be 
explained in Sect. 4.1. Section 4.2 introduces the basic definitions used for the 
description of random variables. Basic methods for the modeling and analysis of 
properties of random variables will be presented in Sects. 4.3 and 4.4, respec- 
tively. Section 4.5 illustrates the application of these methods for the analysis and 
modeling of real data by considering measurements of atmospheric velocities and 
temperatures. The basic features of the analysis and modeling of randomness pre- 
sented in this chapter will be summarized in Sect. 4.6. 

It may be helpful to clarify the use of the terms random and stochastic (which 
are usually used with the same meaning) applied here and in the following. The 
term random will be used for talking about random data (the randomness of data) 
and random variables. The term stochastic will be used for talking about stochas- 
tic modeling, a stochastic process and a stochastic state (which means a stochastic 
process at a certain time). 



4.1 Motivation 



Problem with Randomness. We need temperature (and other) measurements 
to validate, e.g., the performance of weather forecasts or climate models. A typical 
example for such measurements is illustrated in Fig. 4.1. This figure shows 100 
data points of the temperature T (in °C) measured in the stationary stably stratified 
atmospheric boundary layer under almost the same conditions. Details about the 
way in which these measurements were performed are given in Sect. 4.5. Figure 
4. 1 indicates that it is impossible to specify only one value for the temperature T 
under these or any other conditions (other measurements reveal the same picture). 
So how is it possible to make use of such random data? 
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Analysis of Randomness. The application of such data requires mathematical 
concepts for their analysis. First of all, we need information about the mean (also 
called expectation) value of random variables and the characteristic amount of 
randomness involved. Another often required sort of information is given by the 
probability for finding certain events (for example, very high or low temperature 
values). The specific problem regarding the calculation of such characteristics of 
randomness is that all such mean values are a function of the usually limited 
number of observations. This leads to the relevant questions of how important this 
dependence really is, and whether it is possible at all to arrive (for a sufficiently 
high number of observations) at conclusions about the statistical properties of 
random variables that can be reproduced by repeated experiments. 

Modeling of Randomness. The analysis of observations usually leads to re- 
sults that are affected by randomness. Therefore, it is helpful to compare results 
derived from data analyses with analytical models. Models represent ideas about 
distributions of random variables. Comparisons between measured and modeled 
probabilities may show whether such notions about the nature of randomness are 
applicable or not. In addition, results of data analyses are always related to a cer- 
tain range, but it is often relevant to know the probability for the appearance of 
values of random variables outside the range of measured values. Such informa- 
tion requires the development of models for the distribution of random variables. 
The question is then on which basis it is possible to develop such models. 

Questions Considered. The following sections in this chapter will address the 
questions regarding the analysis and modeling of random variables described in 
the preceding two paragraphs. There are two types of random variables: discrete 
and continuous variables. The difference between them is that discrete variables 
take on only certain values, whereas continuous variables may take on any values. 
For example, the results of tossing a coin (head or tail) or rolling the dice (num- 
bers one to six) are discrete variables. Examples for continuous random variables 
are the human height and weight. The mathematical concepts for the analysis of 
discrete and continuous variables are very similar. In this and the following chap- 
ters we will only consider continuous random variables because they are the varia- 
bles that are observed with regard to most problems in nature and technology. 



4,2 Probability Density Functions 

First, let us introduce and illustrate the basic concept used for the description of 
properties of random variables: a probability density function (PDF). This section 
provides the basis for the modeling and analysis concepts presented in Sects. 4.3 
and 4.4. 
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4.2.1 Probability Density Functions 

Mean Value. We consider a random variable X, e.g., the intelligence quotient. 
There are A'^ measurement values X^, where / = 1, A'^ (i.e., we know the intelligence 
quotients of A'^ people). A basic characterization of the random variable X is given 
by the mean <X>, which tells us which value we may expect for X. Therefore, the 
mean is often called an expectation value. To define <X> we use the definition 
applied in Chap. 2, this means the mean value <X> of i = \, N values of any 
variable Xi is given by 

(^)-^l^,. (4.1) 

A relevant question related to the definition (4.1) concerns the number of samples 
A' considered. The influence of A' on the calculation of means will be considered in 
detail in Sect. 4.4.2. Here, we assume that A' is sufficiently large. 

Moments and Central Moments. In generalization of the definition of a mean 
value given by Eq. (4.1), we define a moment of «"' order (« = 1, 2, . . .) by 

X") = ^tx!'. (4.2) 

The setting n= \ recovers Eq. (4.1), which means that a mean is the moment of 
first order. Usually, it is very helpful to analyze random variables by considering 
the mean and deviations from the mean 

X,=X,-{X), (4.3) 

which are called fluctuations. Such deviations from the mean can be characterized 
by the central moment (or the moment about the mean) of «* order, 

X") = J^Y^[X.-{X))". (4.4) 

A central moment of n* order can be expressed by the first n moments by using 
the binomial theorem: see Eqs. (2.23) as an illustration. The definition (4.4) shows 
that the variance defined by Eq. (2.22b) is the central moment of second order. 
There is the obvious question of which information about random variables is 
given by the central moments of an order higher than 2. This question will be 
addressed below in conjunction with the discussion of several probability density 
functions. 

Distribution Function. Information about the moments is usually very helpful, 
but such knowledge does not provide insight into the probability of finding certain 
events (e.g., the probability to have a person in a certain group of people with an 
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intelligence quotient higher than 180). Probabilities can be defined by means of 
theta functions, which are also called step functions or Heaviside functions. The 
theta function 0(z) of any variable z can be defined by 

fO ifz<0 

d(z) = l .. ^^. (4.5) 

[1 if z > 

We may replace z hy z ^ x - X^. Here, x represents any parameter, and X, is one 
measured value of a random variable. Then we obtain 

fO ifx-X,. <0, i.e.,ifx<X 

6'(x-X,)= , ' ' , ^4g^ 

' [l ifx-X,>0, i.e.,ifX,<x 

The probability for finding a value Jf^ < x is one if J^ < x, and zero otherwise. This 
probability is reflected by the theta function 9{x - X^. We have to take the mean 
value of ^ (x - X^ in order to find the probability P{X < x) to find any X<x, 

P{X < x) = \-J:9{x - X,) = {9ix - X)). (4.7) 

To simplify the notation it is usual practice to introduce the distribution function 
(or cumulative distribution function) 

7^(x) = P{X < x) = {9{x - X)). (4.8) 

It is relevant to see here the difference between X and x. X is a random variable 
that is measurable, whereas the so-called sample space variable - oo < x < oo is 
used to analyze the probability of lvalues. Similar to Eq. (4.8), the probability 
P{x <X<x + zix) to find an X value between x and x + Ax is given by 

P(x < X < X + Ax) = {9{x + Ax-X)-9{x- X)) 

^P(X<x + Ax)-P(X<x)^F(x + Ax)-F(x). 

Here, Ax is any positive interval that may be small or not. 

Distribution Function Properties. The properties of the distribution function 
F(x) can be seen in the following way. The probability P{x <X<x + Ax) has to be 
non-negative according to its definition: for each data point X^, the difference of 
theta functions 9{x + Ax - X^ - 9{x - X^ can only be zero or one such that the 
mean of this difference has to be non-negative. Equation (4.9) then implies that 

7^(x) < 7^(x + Ax). (4.10) 

Thus, F(x) increases with x. The minimum of the function F(x) = <9{x - X)> is 
found for F{-co). We find 

7^(-oo) = (4.11) 
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because the minimum of the theta function is zero. The maximum ofF{x) is 

7^(00) = 1 (4.12) 

because the maximum of the theta function is one. Therefore, F(x) is bounded by 
zero and one, 

0<7^(x)<l. (4.13) 

Probability Density Function. A probabihty density function (PDF) is defined 
as a derivative of the distribution function F(x), 

dx \ dx I 

Here, the definition F{x) = <9{x - X)> is used to obtain the last expression. The 
meaning of the PDF is thaty(x) dx determines the probability for finding Xin an 
infinitesimal interval between x and x + dx, 

P(x<X<x + dx) = F(x + dx) - F{x) = fix) dx, (4. 15) 

which follows from the Taylor expansion of 7^(x + dx) for dx — > 0. Knowledge of 
j{x) enables the calculation of all probabilities, for example 

P{a<X<b) = \f{x)dx = j^^^c/x = F{h) - F{a). (4.16) 

The consistency with Eq. (4.9) can be seen by applying x = a and x + dx = b in 
Eq. (4.9). This formula can be used for the calculation of the distribution fiinction 
F{x) for any giveny(x). By replacing a and b by -00 and x, respectively, we find 

F{x) = F{-^)+ ]f{y)dy = ]f{y)dy, (4.17) 

— X' —00 

where 7^(-co) = is used. 

PDF Properties. The PDFy(x) has the following relevant properties, 

/(x)>0, (4.18a) 

/(-«) = /(«) = 0, (4.18b) 

j/(x)^x = l, (4.18c) 

\gix)fix)dx = {g{X)). (4.18d) 

The first property is implied by the fact that/(x) is the derivative ofF{x), which is 
a nondecreasing function of x. Thus,y(x) cannot be negative. The second property 
results from the fact that F{x) tends monotonically to zero or one as |x| — > 00. 
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Hence, the derivatives^) = dF I dx tends to zero. The third property is a conse- 
quence of Eq. (4. 16) for a = -oo and b = co. Because of 7^(co) = 1 and 7^(-co) = we 
find F{co) - F{-co) = 1. The last property applies to any function g(X). We may 
use, for example, g ^ I, g ^ X, or g ^ X^. For g = 1, Eq. (4. 18d) recovers the 
relation (4.18c). The validity of Eq. (4.18d) for any function g(X) can be seen by 
rewriting the left-hand side of Eq. (4.18d), 

jgix) f{x)dx = \g{x) — ^— '-dx = — Zjg(x) '-dx 

dx N ,=1 dx 



1 ^ 

=— z 



1 " 
=— z 



y dX,^ 
dX,, 



\gix)eix-X.)dx (4.19) 

}g(x)«fx = ^£g(X,,) = (g(X)). 

X: Jy 1 = 1 



The first line makes use of the definition (4.14) ofJ{x) and the definition of mean 
values. In the next line, the derivative by x is replaced by the derivative by -X^. 
Then, we reduce the integral to the range x > Xj for which 0(x - X^ > 0, where 
Z — > 00. Finally, we make use of the Leibnitz integral rule to recover <g{X)>, 



^igix,t)dt = / ^i^dt + g{x,h(x))^-g{^M^))^- (4.20) 

Here, the derivatives are assumed to exist. 



4.2.2 Delta Functions 

The theta function was used for the representation of the distribution function 
F{x). The PDFy(x) = <d0(x - X)l dx> was introduced as mean of the derivative of 
a theta function. Let us have a closer look at the properties of step functions and 
their derivatives, which will be helpful for the work with PDFs. 

Theta and Delta Functions as Limits. Let us consider first the function On (^) 
defined by the expression 

^.(x).i±i5f^. (4.21) 

Examples for 0^ (x) are shown in Fig. 4.2. This figure shows that 9^ (x) tends for 
large A^ to the theta function 

fO ifx<Ol 

^(x)=lim^„(x)= . (4.22) 

A'^" 1 ifx>0 
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Fig. 4.2. The functions 9^) (first row) and 5^ (second row), which are defined by 
Eqs. (4.21) and (^^IT), respectively, are shown for the given values of A^. 



To obtain an expression for dO {x - X)l dx that we can use to see the properties of 
this derivative we introduce the function 



Sfiix)' 



d9f^ (x) 
dx 



(4.23) 



This relation does also apply to the limit that A^ — > oo. In terms of Eq. (4.23) we 
obtain for the delta function 5(x) (or Dirac delta function or Dirac function) 



5{x)= lim(J^(x): 



lim 

yV->oc 



dOf^ (x) d 



dx 



lim6'^(x): 

dx i^^" 



dd(x) 
dx 



(4.24) 



The last expression arises from the definition (4.22) of the theta function as limit 
of Ofj(x) forN^>^ 00. The advantage of introducing Sfj(x) is that we can obtain now 
a representation of S(x) = d9{x-X) I dxm terms of the limit 



S{x) = lim Sm (^) - lim 



dOf^ (x) 
dx 



= lim z . 

^'^"2cosh^(A^x) 



(4.25) 



The last expression can be found by taking the derivative of 0^, (x) defined by Eq. 
(4.21). Examples for dj^ (x) are given in Fig. 4.2. This figure shows that S{x) 
vanishes for all x ^ 0, and it diverges for x — > 0. For that reason, the delta function 
is referred to as a generalized function or a distribution. 
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Properties. A generalized function is a function that does not have to exist for 
all arguments, but integrals over such functions (multiplied with other functions) 
have to exist. The delta function has the property that it integrates to one, 

\5{x)dx=\ — tfc = 6'(oo)-6'(-oo)=l. (4.26) 

dx 

The most useful property of the delta function is its so-called sifting property, 

\g{x)S\x - a)dx = g{a)\5\x - a)dx = g{d). (4.27) 

Here, g(x) can be any function of x, and a is any finite real number. Justification 
for the middle expression arises from the fact d{x - a) vanishes everywhere except 
at X = a. Therefore, g(x) can be replaced by g{a) that is independent of x. The last 
expression results from the normalization property given by Eq. (4.26). Another 
relevant property of the delta function is given by 

j g{x) 5{- x)dx = \ g(x) 5{x)dx . (4.28) 

The right-hand side is equal to the left-hand side because the only nonzero contri- 
bution of the delta function is at x = 0. There are many other properties of delta 
functions - useful additional information about delta functions can be found else- 
where (Heinz 2003). 

PDF Definition. According to the PDF dQfimiion J{x)^<d 6 {x-X) I dx>, the 
expression S{x) = d9{x-X)l dx can be used for the following definition of a PDF, 

f{x) = {5{x-X)). (4.29) 

This definition can be used to recover all the PDF properties. Property (4. 1 8a) is a 
consequence of the fact thaty(x) is the mean of a non-negative variable. Property 
(4. 1 8b) is caused by the fact that the delta function vanishes at infinity, 

/(±oo) = ((?(±oo - X)) = 0. (4.30) 

Property (4.18c) is implied by the normalization of delta functions, which is given 
by Eq. (4.26), 



\f{x)dx = \{5{x-X))dx = l\5{x-X)dx\ = (l) = 1. (4.31) 

Property (4.18d) is implied by the sifting property (4.27) of the delta function, 
\g{x) f(x)dx = jg(x) {S(x - X))dx = i\gix) 5{x - X)dx] 



(4.32) 
= {\g(X)S(x-X)dx) = lg(X)\Six-X)dx) = {giX)). 
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Fig. 4.3. An illustration of a uniform PDF that has a 
constant value l/{h-a) between a and b. The mean 
value <X> = {a + b) / 2, and the standard deviation 



<X'>"'-ib-a)/{2V'^). 



4.2.3 An Example: The Uniform Probability Density Function 



Uniform PDF. It is relatively often the case that there is no information about 
the distribution of a random variable. In this case, it is reasonable to assume that 
there is a uniform probability for all values of this random variable inside a certain 
range of variations. The model for such a PDF is given by a uniform PDF, 



fix)- 



\l{b-a) 




\i a<x<b 
otherwise 



(4.33) 



Here, the parameters a and b define the range of nonzero probability. This PDF 
satisfies the normalization condition to integrate to unity. 



1 * 

\ f{x)dx = \dx - 1. 

b-a „ 



(4.34) 



Probability. To calculate the probability to findXbetween any two bounds we 
calculate first the distribution function F{x). By means of Eq. (4. 17) we find 



FW^J/W^.J "'"-^-'"-^-' ^: 



min(x, b) - min(x, a) 



(4.35) 



Here, the PDF is represented in terms of a difference of theta functions that is only 
nonzero if a < y < b. The use of P{c < X < d) = F{d) - F{c) then results in the 
following expression for the probability to find X between c and d, 

mm{d,b) - mm(d,a) - mm(c,b) + min(c,a) 



P{c<X<d) = F{d)-F{c) = - 



Two examples for the use of this expression are the following ones: 



P{c<X<d) 



b- 


- a- 


- c + c 


d 


b- 
- a 


-a 
- c + a 




b- 


- a 



= 1 



ifc<a<b<d 
ifa<c<d<b 



(4.36) 



(4.37) 
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Moments. The mean value of a uniform PDF/(x) is given by 



1 



(X') = \x f(x)dx = \xdx = 

^Z-' b-ai 2 b-a 



Ib^-a^ l(b-a){b + a) a + b 



(4.38) 



Therefore, <X> is the mean position between a and b, see Fig. 4.3. The central 
moments can be calculated in the following way {k= 1,2, . . .), 



X' 



-.\[x-{x)y f{x)dx- 



\\ X I dx 



1 



{k + \){b-a) 
1 2 



b-a 

A'+l 



(4.39) 



2ik + l){b- a) 



b-a 



■(-1)* 



A+ii b-a 



l-(-l)^' 
2{k + 1) 



Hence, all the odd central moments disappear {k= 1,2, . . .), 



X 



Ik-l 



Q. 



The even moments are detennined by the formula {k^ 1, 2, ...) 

^2* o/f 



X 



1 



2k + \ 



- a 



3" 



2k + 1 



X' 



The last expression was obtained by adopting the second central moment 

,2 



X' 



I b-a 

31 2 



According to this expression, the standard deviation < X' >"* is given by 

^2\"2 b-a 



X' 



2S 



(4.40) 



(4.41) 



(4.42) 



(4.43) 



Here, we assumed b> a. <J^^>"^ characterizes the typical amount of deviations 
from the mean. The standard deviation scales the width of the PDF, but there is no 
need that < X^ >"' is equal to the PDF width (see Fig. 4.3). 

Parameters. For given values of the mean and standard deviation we can cal- 
culate the model parameters a and b. According to Eqs. (4.38) and (4.43) we have 

b + a = 2{x), b-a = 2S{x^)'^\ (4.44) 

Hence, a and b are determined by the mean and standard deviation according to 

b = {X) + 43(x^)"\ a = (X)-V3(l^y", (4.45) 

as may be seen by taking the sum and difference of a and b. 
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Random Number Generation. The relevance of the uniform PDF also arises 
from the fact that random numbers having a uniform PDF, which can be generated 
in several ways (L'Ecuyer 1994), can be used to generate random numbers having 
different continuous distributions. To show this, we consider a random variable X 
that is uniformly distributed over the interval [0, 1], i.e., the PDF of Xis given by 

fl if < X < 1 

fix) = \ . (4.46) 

[0 otherwise 

The corresponding distribution flinction of X is given by 

F{x) = P{X <x)=] f{z) dz = ] [d{\ - z) - 0(0 - z)] dz = min(x, 1) - min(x, 0). 

—X) —CO 

(4.47) 
Here, the PDF J{x) is written in terms of the bracket term, which is only nonzero 
for < z < 1 . The latter result can also be written 



Fix)^ 



ifx<0 

X ifO<x<l. (4.48) 

1 ifx>l 



We are interested to generate random numbers that have a specified distribution 
function G(y). Then, it turns out that the random variable 

Y^G-'(X), (4.49) 

which can be determined by solving the relation G{Y) ^ X for Y, has the given 
distribution function G{y). The correctness of this claim can be shown in the 
following way. The distribution function of 7= G'\X) is given by 

F(y) = PiY <y) = p(g-' (X) <y)= P{x < G(y)). (4.50) 

The last expression arises from the fact that F(y) is a monotonic function. Thus, 
the inequality G '(X) < y is satisfied if and only ifX< G(y). P(X < G(y)) can be 
calculated by means of the known distribution function of X. By replacing x in 
Eq. (4.48) by G(y) we obtain 



F{y) = PiX < G(y)) - 



ifG(j)<0 

G(y) ifO<G(>')<l. (4.51) 

1 ifG(j)>l 



Due to the fact that < G(y) < 1 , this relation implies that 

Fiy) = Giy). (4.52) 

Therefore, the random variable Y has the given distribution function G(y). This 
method is called the inverse transformation method (Ross 2010). 
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Fig. 4.4. The PDF f(x) of a uniformly distributed 
random variable x. Here, L = b - a refers to the 
interval in which the PDF is nonzero. 



4.3 Models for Probability Density Functions 

After defining PDFs and discussing their basic properties, let us consider the 
question of how it is possible to find specific PDF shapes for certain observations. 
There are two main approaches to address this question: we can design PDFs on 
the basis of any principle (e.g., the constraint considered below that the PDF has 
to maximize the uncertainty), or we apply empirical PDF shapes that have desired 
properties. The first approach will be explained in Sects. 4.3.1 and 4.3.2, which 
includes the discussion of the most commonly used PDF - the normal PDF. This 
approach will be presented for unbounded variables that may have values between 
negative and positive infinity. The second approach will be used in Sects. 4.3.3 
and 4.3.4 to obtain PDFs for the relevant cases that the variables can only be 
positive and that the variables can only vary between zero and one, respectively. 



4.3.1 Statistically Most-Likely Probability Density Functions 



Predictability. For the development of PDF models it is helpful to relate the 
shape of a PDF to a measure that characterizes the predictability of the state of a 
random variable. The predictability of states is illustrated in Fig. 4.4, which shows 
that the predictability of the state of a random variable depends on the PDF shape, 
which is controlled hy L = b - a. For Z — > 0, the PDF becomes a delta function. 
The predictability is maximal in this case (the uncertainty is minimal): we know 
that X will realize the value at the location of the delta peak. For Z — > co, the 
predictability of the state ofXis minimal (the uncertainty is maximal): there is the 
same probability for all possible states. The consideration of the predictability of 
states of random variables can be used in the following way for the construction of 
PDFs. Very often we do not know anything about a PDF with the exception of a 
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few low-order moments (means and variances), for which reasonable estimates are 
often available (contrary to measurements of moments of higher order that are 
usually significantly affected by randomness). However, knowledge of a few low- 
order moments is insufficient to determine the PDF shape, which requires infor- 
mation about all the moments. What we can do in this case is to apply information 
about the known moments combined with the constraint that the predictability 
(uncertainty) related to the PDF has to be minimal (maximal). The latter approach 
corresponds to the following idea (Du et al. 1994). First, we reduce our uncertain- 
ty by the given information (the known moments). Second, we are maximally un- 
committed with respect to the missing information (the PDF shape). The use of 
this concept will be shown in the following. Interestingly, this approach can be 
extended to develop PDF models for several variables, and it can be also used for 
boimded variables (see the exercises 4.3.5 - 4.3.8). 

Measure of Uncertainty. The realization of this concept requires the definition 
of a measure S of the uncertainty of the state of a random variable. This measure 
of uncertainty S, which is called entropy, is defined by 

S = -jfix)ln{f{x))dx. (4.53) 

Instead of justifying this definition of 5' in detail (Shannon 1948, Jaynes 1957), let 
us illustrate the suitability of this definition with regard to a uniform PDF. The 
combination of the definition of 5 with the uniform PDF shape (4.33) shows that 
the entropy of a uniform PDF is given by the expression 

S = ^—\lJ^—]dx = ^^^^^^{b-a) = lnL, (4.54) 

b- a ^ yb - a ) b - a 

where L = b - u'k used. This expression illustrates the suitability of using 5 as a 
measure of uncertainty: the uncertainty S is minimal for Z — > 0, and the uncer- 
tainty is maximal for Z — > co. 

Statistically Most-Likely PDF. We will construct now a PDF according to the 
concept described above. We assume that we know « = 1, ■■-,$ moments 

X")^\x"f{x)dx. (4.55) 



The goal is to construct a PDF that has s moments that agree with the given ones 
but maximizes the entropy S (i.e., the uncertainty). According to the calculus of 
variations, we extend the entropy 5 to a functional S by involving Eq. (4.55), 

5' = -j{/Wln(/(x))+ t//,x' fix) - fix)\dx. (4.56) 

The //j- are Lagrange multipliers which have to be chosen such that Eq. (4.55) is 
satisfied for all n. The last term on the right-hand side of Eq. (4.56) modifies the 
multiplier //q, which simplifies the following explanations. To fmd the maximum 
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of 5', we calculate the functional variation of 5* with regard to/ 

^ = -d]n{fix))+±M,Adx. 
q [ k=Q J 

This functional variation is equal to zero if the PDFy(x) is given by 

/(x) = expi-X//,x* 



(4.57) 



(4.58) 



The PDF (4.58) is called a statistically most-likely (SML) PDF. By introducing 
nondimensional Lagrange multipliers If, we can also write the latter relation as 



/(x) = exp- 



^-z 



k[--{x)Y 



X- 



(4.59) 



The s + \ factors /Ij. are uniquely determined by the normalization of/(x) and the s 
conditions (4.55). These conditions for the PDF (4.59) can be written 



X")^\[x-{X))" f{x)dx, 



(4.60) 

where n =0, 1, ■■■, s. Evidence that the PDF (4.59) maximizes the entropy is 
provided in terms of the exercise 4.3.1 by the comparison of the entropy of a SML 
PDF with the entropy of the corresponding uniform PDF. 



4.3.2 The Normal Probability Density Function 

Second-Order SML PDF. The normal PDFy(^) of one random variable X is a 
second-order SML PDF. This PDF is given by 



/(x) = exp 



/Iq -/I, 



W ^M^ 



X- 



X' 



(4.61) 



The parameters X^, X^, and X^ have to be chosen such that the PDF (4.61) satisfies 
Eq. (4.60) for « = 0, 1,2, 



= j(x-(jr))/(x)fl'x, 
{X^) = \[x-{X))' f{x)dx. 



(4.62a) 
(4.62b) 
(4.62c) 



for any given mean <X> and variance <X^>- 
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Fig. 4.5. The normal PDF/(x) given 
by Eq. (4.72) with mean // = 1 and 
standard deviation o" = 1 . 



Parameter Calculation. A simple way to calculate the model parameters is to 
differentiate y(x), 



df _ f 

dx I y'^ 



- A, — Aj 



X- X) 



X- 



(4.63) 



The integration of this relation leads to 



[df , If 



dx 



X' 



L +/ln 



x-iX) 



X' 



fdx = - 



K 



X' 



(4.64) 



where Eqs. (4.62a) and (4.62b) are applied. The left-hand side disappears because 
the PDF j\x) disappears at infinity. Therefore, we find the requirement X^ = 0. 
By multiplying Eq. (4.63) for Ai = with x-<A> and integration we find 



1^, 
dx 



j(x-(X))|-^x = -^j(x-(X))V^--^,, 



(4.65) 



where Eq. (4.62c) is used. The left-hand side can be written 



\[x-{x)YUx = \ 



d{x-{X))f d{x-{X)) 



dx 



dx 



f 



dx 



[(^-W)/L-[(^-W)/L-!/^^--i- 



(4.66) 



The term (x- <X>)/is zero at positive and negative infinity, see Eq. (4.61). The 
combination of the last two relations leads then to the conclusion that /Ij = 1 . The 
parameter /Iq can be calculated by means of the condition (4.62a), 



l = e^jexpf itJ^^^x, 



X' 



(4.67) 
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where the findings /I, = and /I2 = 1 are applied. Now, we introduce the variable 
J = (x - <X>) /{2<X^ >)"^ to simplify the latter relation, 



-''■0 



h{X^^\e-'~dy. (4.68) 

To calculate the integral involved it is helpful to consider the squared integral 



[j 



e * dy 



■■\e-'' dy je-'' dz = | je-<'''+-"'' dydz. (4.69) 



The right-hand side integral can be calculated by introducing the polar coordinates 
J = r cos 6 and z = r sin 6. In terms of dy dz ^ r dd dr we obtain 

[\e-''dy 



= \ \ e-'" rdddr^lTTJ re-'" dr = -tt \ — — dr = tt. (4. 70) 

00 dr 



The combination of this result with Eq. (4.68) then shows that 

e-'^=^2n(x'-). (4.71) 

The second-order SML PDF (4.61) that follows from the latter result and /I, = 
and 2o_= \ can be written in terms of standard notation as 

fix) = ^^expj- ^^^1. (4.72) 

-nna [ ley \ 

Here, the model parameters // and care given by 

// = (^), a = (l^)"'. (4.73) 

The second-order SML PDF (4.72) is the normal PDF. The PDF/^) is illustrated 
in Fig. 4.5. This figure shows the relevance of the mean: <X> = // describes the 
expectation value, which is here the peak value of the PDF. The square root of the 
variance a= < Jf^>"^ is the standard deviation. Figure 4.5 shows the meaning of 
the standard deviation: this parameter characterizes the intensity of fluctuations 
(the amount of randomness). In correspondence to the properties of the uniform 
PDF we observe that a does not determine all the range of fluctuations, but it 
defines the order of magnitude of fluctuations. The PDF is a symmetric function 
about the mean. Thus, all the odd central moments disappear (k= 1,2, ...), 

i^'*"'\ = 0. (4.74) 

The even central moments are determined by the formula {k= 1,2, ...) 

~2A\ (2^^2.^ (4.75) 
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Skewness and Flatness. To compare any PDF with a normal PDF, or to see 
whether a random data set can be described by a normal PDF, it is helpfijl to use 
normalized moments of third-order (the skewness) and fourth-order (the flatness 
or kurtosis) as a reference. These nondimensional numbers are defined by 

-3/2 I ~ .\ / ~,\-2 



X^') X 



m. 



X') X' 



(4.76) 



The skewness m^ indicates deviations from the symmetry of fluctuations about the 
mean value. According to Eq. (4.74), one finds for the normal PDF 

m, = 0. (4.77) 

The flatness 111^ indicates the peakedness of a PDF. According to Eq. (4.75), a 
normal PDF is characterized by 

m^=3. (4.78) 

A PDF with m^>3 {m^ < 3) has a higher (lower) peak value than the normal PDF. 
It is worth noting that rrin and iri} cannot take any values: they have to satisfy the 
inequality mj^ + I <m^ (see Eq. (2.38)). 

Probability. The probability P{a < X < h) for finding the random variable X 
between a and h is defined by Eq. (4.16). For a normal PDF we find 

n2 " 



P{a<X<b) = lf{x)dx = 



1 



-Jexp 



{x-mY 



dx. 



1271 o-'a ' I 2a' 
We introduce J = (x-//)/(2"^ a) to simplify this expression, 

1 ^ 2 
Pia<X<h) = -i^je-' dy. 



(4.79) 



(4.80) 



where the bounds are given hy A ^ {a- /j)/ (2"^ a) and B = {h- ju)l (2"^ a). The 
integral (4.80) cannot be explicitly calculated. The most convenient way to deal 
with this problem is to write P{a <x<h)'\n terms of a standardized integral. To 
prepare this representation we write P(a <x<b)d& 



P(a<X<b) 



1 



yjn 



J e dy + ^e dy 



1 



fe ' dy - \e ' dy 



The latter expression can be written 

P{a<X <h) = ^[erf{B)-erfiA)\, 
where the error function erf{x) is defined by the expression 

2 } _2 

erf{x) = —i=j e ' ds. 



(4.81) 



(4.82) 



(4.83) 
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Fig. 4.6. (a) The error function erfix) defined by Eq. (4.83) is given as a solid line, 
the approximation (4.84) as a dashed line (no noticeable difference between these 
curves); (b) the relative error e (in %) of the approximation (4.84). 



Error Function Approximation. The error ftinction erj{x) can be numerically 
calculated in terms of its integral definition (4.83). However, the use of an analyt- 
ical approximation is much more convenient. Such an approximation is given by 



E{ 



x) = ±i 



Here, the abbreviation H is defined by 



H 



4/n: + px 



\ + px 
and the parameter/? is given by 

^ ""'^ 0.1400. 



(4.84) 



(4.85) 



(4.86) 



3;r ;r-4 

The positive sign in Eq. (4.84) applies to positive x values, and the negative sign 
applies to negative x values. Figure 4.6 shows that the approximation (4.84) is 
very good: the magnitude of the relative error of this approximation is smaller 
than 0.04%; the figure also reveals the relevant property erj{-x) = -erj{x) of the 
error function and of the approximation £'(x) of the error function. 

Example Probabilities. Let us calculate some probabilities to illustrate the use 
of Eq. (4.82) combined with the approximation (4.84) for the error function. 



P(// -a<X</j + (j) 



e//(l/2"^)-er/(-l/2"^) 



:e,/(2-"') = 0.683, 



P{/u -2(T<X</u + 2(t)- 



erf{2l2"')-erf{-2l2"') 



:er/(2/2"') = 0.955, 



P(// - 3cr < X < // + 3cr) : 



e./(3/2"^)-er/(-3/2"^) 



:e//(3/2"') = 0.997. 



(4.867) 
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Fig. 4.7. A normal PDF f{x) with mean /J= \ and standard deviation a= \. (a), 
(b), and (c) illustrate the probabihties P(//- a<X<ju+ o) = 68.3%, P{ju-2a<X< 
/u+2a) = 95.5%, and P{/j - 3 a<X</j+ 3 a) = 99.7%, respectively. 

A and B are determined hy A ^ (a- ju)/ (2"^ o) and B = {h- jJ)! (2"^ <t), and the 
property erj{-x) = -erfix) was used here to simplify these expressions. The corre- 
sponding probabilities are calculated in tenns of the approximation E{x) of the 
error function. Figure 4.7 illustrates the relevance of these probabilities. The prob- 
ability for finding X between ju- 3 a and ju + 3 a is close to one. Consequently, the 
magnitude of almost all fluctuations will be smaller than 3 a. A mean value that 
characterizes both small and large fluctuations is given by the standard deviation 
<X^ >"^ = a. Thus, cris used as a measure for the typical intensity of fluctuations. 
The probability to find X between jU- a and ju+ a is about 2 / 3 . 

Application of the Normal PDF. The normal distribution is used for a variety 
of applications, for example the distribution of 

• intelligence quotients, where //= 100, a^ 15 (Bulmer 1979, Stewart 2006) 

• heights of adult males in the U.S., where //= 1.75 m, 0"= 0.07 m (Stewart 2006) 

• lengths of human pregnancies, where // = 268 days, 0"= 15 days (Stewart 2006) 

• test scores, where, e.g., jU = 80%, a= 10%i 

Other examples for normal PDFs will be shown below. 



4.3.3 The Gamma Probability Density Function 



There are many circumstances under which the application of a normal PDF is 
inappropriate. One example is given by the relevant case that the random variable 
is non-negative. The latter case is given, e.g., regarding the atmospheric raindrop 
size distribution and the droplet size distribution in a cloud (both distributions are 
relevant to climate modeling). A way to handle such cases will be considered now 
by the following discussion of the gamma PDF. 
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Fig. 4.8. The solid and dashed lines in (a) show the gamma function (4.89) and its 
approximation (4.90), respectively. There is hardly any difference between both 
curves; (b) shows the relative error e (in %) of the approximation (4.90). 



Gamma PDF. The PDF of non-negative random variables is often modeled by 
the gamma PDF. This PDF is defined by 



/W^ 



r{a){x) 



bx 



exp 



hx 

M 







ifO<x 
otherwise 



(4.88) 



The idea of this PDF is to consider an exponential ftinction ey.^{-b x I <X>) that is 
multiplied with a power ofbxl <X> to increase the flexibility of PDF shapes (to 
cover more scenarios that can be described). The PDF parameters a and b are non- 
dimensional. r(a) is the gamma function that is defined by the integral 



r{a) = \y" 'e -dy. 



This integral cannot be solved analytically, but it can be approximated by 



r{a) = 



asinh — | + - 



-Nfl 



810fl' 



(4.89) 



(4.90) 



Figure 4.8 shows that Eq. (4.90) approximates the gamma function very well. For 
example, the relative error at a = (1, 2) is e = (0.0183, 0.0003)%, respectively. The 
gamma function has the relevant property 

r(a + l) = ar{a), (4.91) 

which can be used to calculate the gamma function at higher a - the accuracy of 
(4.90) increases with a. The latter property can be proven by integration by parts. 



! 



dy''e 
dy 



-dy = = a\y" 'e -'dy - \y"e 'dy. 



(4.92) 
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The reason for the appearance of the gamma function in Eq. (4.88) can be seen by 
proving that the gamma PDF satisfies the normalization condition, 



jf(x)dx-- 



r(a){X' 



-\ 



ox 



X) 



exp 



(X) 



dx ■■ 



1 



n«)i 



\y^-'e-'dy = \, (4.93) 



where the substitution >> = bxl<X> was applied. 

Moments. The moments of fix) can be calculated in this way {k= 1,2, . . .), 



X') = \x'f{x)dx 



X) 



r{a){x) b' 



-\ 



bx 

Jx) 



Yf 



X) 



exp 



bx 

Jx) 



dx 



(4.94) 



Fia) b' i b' r{a) 



Here, y = bxl <X> and the definition (4.89) of the gamma function were applied. 
The ratio of gamma functions, which is often denoted by Pochmammer's symbol 
(a),, = T{a + k)IT{a), can be rewritten by making use of r(a + 1) = a T{d), 



r(a + k) (a + k-l)r(a + k-r) ia + k-l)(a + k-2)r(a + k-2) 



r(a) r{a) 

- (a + k — l)(a + k — 2)---a. 

Hence, the moments of fix) are given by 



^i \ {a + k-l)(a + k-2)---a , > a 



r(a) 



(4.95) 



(4.96) 



Parameters. The last expression shows that the moments off{x) are functions 
of the parameters a and b. These relations can be used for the calculation of a and 
b as functions of the mean and the variance, which enables the adjustment of the 
gamma PDF to any given values for the mean and the variance. According to 
Eq. (4.96), the mean is given by 



X) 



X) 



(4.97) 



and the second-order moment is 



x'] = i^^^(x)\ 



(4.98) 



The variance <X^ > is related to these moments by <X^ > = <X^> - <X>^, see 
Eq. (2.23a). Applying the last two relations, we obtain for the variance 



X' 



^(x)\ 



(4.99) 
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Fig. 4.9. An illustration of gamma PDFs for <X> = 1. The standard deviation 
<X^ >i/2 = 2, 1, 0.5, 0.25 in (a), (b), (c), and (d), respectively. 



The obvious requirement of Eq. (4.97) is that a^h. Hence, we find 
a=b ■■ 



X' 



(4.100) 



which enables the calculation of a = Z; for a given mean and variance. By using 
a^ bwe can write the gamma PDF (4.88) for x > as 



fix)- 



ax 



r{a){X) 



X) 



exp 



ax 



X) 



(4.101) 



Non-Normality. The skewness m^ and flatness m^ implied by the gamma PDF 
are given by the expressions 



Vfl 



/W4 =3| 1 + - 



(4.102) 



see exercise 4.3.9. The parameter a is positive due to Eq. (4.100). Thus, for finite 
values of the parameter a we find that both m^ and m^ are unequal and bigger than 
the corresponding values of a normal PDF. The values m^ = and 7W4 = 3 for a 
normal PDF are recovered in the limit that a — > co. 
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Illustration. Figure 4.9 illustrates the broad range of variations of the gamma 
PDF. The value a = 1 specifies a particular PDF, the exponential PDF 



/W^ 



J_ 



-exp 



f 


X 


V 


(^)J 



(4.103) 



for which we have r(l) = 1. The behavior of the gamma PDF is different for the 
cases that a < 1 and a> \. For a < 1, the gamma PDF looks like a modified expo- 
nential function. For a > 1, the PDF looks like a modification of a normal PDF. 
The nonnal PDF features m^ = and 171^ = 2 are recovered for a finite mean value 
if the variance < X^ > disappears, < X^> ^Q, which implies that a — > oo. 

Probability Calculation. Unfortunately, the calculation of the probability of 
events cannot be performed analytically on the basis of the gamma function. A 
way to handle this calculation is to follow the approach used for the calculation of 
the integral over the normal PDF. The application of the gamma PDF (4.101) in 
the definition (4. 16) of P(c <X<d) leads to 



P{c<X <d) 



r{a){X 
1 



f \ 
ax 



X) 



exp 



ax 



dx ■■ 



1 



Fia) 



\y'''^e'''dy 



Fia) 



jy" 'e '' dy-\y" 'e - dy 



(4.104) 



F,{a,D)-r,{a,C) 
Fia) 



Here, we applied the substitution y ^ ax/ <X> and the bounds C ^ acl <X> and 
D = adl<X>. We did also use the definition of the incomplete gamma function 



F,{a,x) = ^y'' e ■ dy. 



(4.105) 



This function recovers the gamma function T{a) for x — > oo. The advantage of 
Eq. (4.104) is that we can use now the series expansion of the incomplete gamma 
function (Abramowitz & Stegun 1984) for the calculation of P(c <X<d), 

(-A-)" 



Fj{a,x)^x''Y, 



{a + n)n\ 



(4.106) 



4.3.4 The Beta Probability Density Function 



Beta PDF. A normal PDF can also not be applied to cases where the random 
variable is non-negative and has a finite range of variations. Such a case is given, 
e.g., regarding the mixture fraction (which represents a measure for the degree of 
the mixedness of a certain substance) that can only vary between zero and one by 
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definition. A gamma PDF cannot be used for this case, too, because it predicts 
nonzero probabilities for finding values of such a variable outside of the allowed 
range of variations. The distribution of such random variables is often described 
by a beta PDF that is defined by 



/w = 



1 a-[ /I nA-1 



x^-'il-x)"-' ifO<x<l 



B(a,b) . (4.107) 

otherwise 



Here, B{a, b) is the so-called beta function. This function is defined as the integral 

B{a,h)^\y''-\\-y)''-'dy. (4.108) 



B{a, h) can be calculated via its relation to the gamma function (Abramowitz & 
Stegun 1984), 

Bia,b) = £}^. (4.109) 

r{a + h) 

The use of the beta function definition (4.108) shows that the beta PDF integrates 
to one, 

1 1 1 

\f{x)dx = jx"-' (1 - x)" dx = l. (4.1 10) 

B(a,b) 

The characteristic features of this PDF (the moments, relations between the model 
parameters a and h with the mean and variance, the skewness and flatness) can be 
analyzed in correspondence to the analysis of the gamma PDF: see the exercises 
4.3.10 and 4.3. 11. 

Illustration. Figure 4.10 illustrates the range of variations of the beta PDF. The 
beta PDF is often used to characterize the mixing of substances, so let us describe 
the beta PDF features with regard to this case. For relatively small values of 
a = h= 0.28, Fig. 4.10a basically describes the existence of two states: one state 
where the random variable X » 0, and a second state where the random variable 
X« 1. After a certain amount of time, mixing leads to the uniform PDF that can be 
seen in Fig. 4.10b. This uniform PDF is characterized by larger a = b = \ values 
that imply m^ = and mn = 9 I 5. Additional mixing leads to the appearance of a 
PDF that is close to a normal PDF: see Fig. 4.10c that is obtained for a = Z; = 5.1. 
The corresponding flatness value m^ = 2.5 is smaller than the value m^ = 3 found 
for a normal PDF. For the case a = Z? — > oo we find a peak value that goes to 
infinity. This case corresponds to a state of perfect mixing where there is only a 
nonzero probability for the value X = 0.5. The PDF is then characterized by the 
values m^ = and /H4 = 3 of a normal PDF. 
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Fig. 4.10. Beta PDFs for a mean of <X> = 0.5. 
The standard deviation has the values < X^ >"^ = 
0.4, 0.5/3 "^ 0.15 in (a), (b), and (c), respectively. 



Probability Calculation. To calculate probabilities P{c < X < d) we apply the 
beta PDF (4. 107) in the probability definition (4.16), 



1 '' 1 

P(c<X<d) = \x"-' (1 - x)'-' dx = 

B{a,b){ B(a,b) 



\x" (1-x)' dx 



-jx" (1-x) dx 



B,(a,b,d) - B,{a,b,c) 
B(a,b) 



(4.111) 



It is assumed here that < c, d < I. In this equation we applied the definition of 
the incomplete beta function 



Bj{a,b,x) = \y'' \l-yf ^dy, 



(4.112) 



which recovers the beta function B{a, b) for x= I. The advantage of Eq. (4. 1 1 1) is 
that we can apply the following series to calculate the incomplete beta function 
(Abramowitz & Stegun 1984), 



B,{a,b,x) = x''Y, 



r(l-b + n) x" 



^ z 



p,,^ 



„=o r{\-b) (a + n)n\ „=oa + n 



(4.113) 



The last expression introduces the abbreviation/?,, = r(l -b + n)/[r{l -b) n\]. The 
definition of/?,, shows that /»o = 1 and /»„ = /»„_, (n-b) / n for n> \. Hence, p„ is 
finite for increasing n. The calculation of/»„ by/»„ = p„-\ (n-b)/ n enables a stable 
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numerical calculation of B^a, h, x): the problem to evaluate r{l -h + n) at large n 
(which appears i{Bj(a, h, x) is calculated for x > 0.9) is avoided in this way. 



4.4 Data Analysis 

Let us address now the question of how simulation results and measurements of 
random variables can be analyzed. A particular problem of such data analysis is 
the question of how means and PDFs derived from data depend on a finite number 
of samples applied. 



4.4.1 Calculation of Statistics 



Filtered PDFs. The definition J[x) ^<d0 (x - X) / dx> of a PDF involves a 
derivative. In order to calculate a PDF from measurements or simulation results, 
we have to represent the derivative in a discrete way. This can be done by using 
for the PDF the expression 



y.„.±.Uf-x-4.-f-. 






(4.114) 



The last expression shows thaty^(x) is a PDF that is filtered over the interval Ax. 
No assumption is made here about the interval Ax, which may be small or not. The 
subscript A refers to the filter operation defined by Eq. (4.1 14). This notation will 
be also used below for other functions that are filtered over an interval Ax. From a 
computational point of view it is helpful to represent /^(x) in another way, 



AW = ^Mx + -^ 



X -a 



Ax 



X ) = 



1 AV 



(4.115) 



According to the definition (4.1) of means, the number zW is defined by 

Ax 



Av=2: 



N 

=z 

1=1 



2 

1-1 = 
1-0 = 1 
0-0 = 



-X, 



-0\x-f-X. 



ifX, <x-Ax/2 

if x-Ax/2<X,. <x + Ax/2 

ifx + Ax/2<X 



(4.116) 
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Consequently, AN measures the number of samples that are found in the interval 
x-Ax/2<Xi<x + Ax/2. The PDF f^{x) = AN/ (NAx) represents, therefore, the 
relative number of samples around x normalized by the filter interval Ax. 

Properties of Filtered PDFs. For any function g{x), the filtered PDF f^{x) has 
the property 



1 gix) /a (x)dx = —n g{x) 
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dx 



-^( ]gix)dx- ]gix)dx\ = ^l lgix)dx+ \gix)dx) (4.117) 

/X+Axl2 



1 X+MI2 \ 

— \g{x)dx) = {g,{X)). 



The first rewriting results from using the definition (4.1 14) ofy^(x). The following 
rewritings make use of the properties of theta functions. In correspondence to the 
expression (4.1 14), the last expression represents the mean of the filtered function 
gi^{X). The relation (4.117) enables some interesting observations that follow from 
the calculation of the integral overg^(x): see the first expression of the last line. 
• By setting g^ = 1 we find Ih&ifJ^x) represents indeed a PDF because it integrates 
to one, 

Ax^ 



|/.W.x.i-X.f 



X- 



= 1. 



By setting g = x we find 



(4.118) 



\xf^{x)dx 



1 



X + 



Ax 



2Ax U 2 

By setting g = {x- <X>y we obtain 
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6X'Ax/2 + 2(Ax/2)' 
3A^ 



X' 



12 



(Ax)^ 



(4.119) 



(4.120) 



What is the relevance of these properties of filtered PDFs? The Eqs. (4.118) and 
(4.119) show that the filtered PDF integrates correctly to one, and it provides the 
correct mean value. Equation (4.120) shows that the variance of the filtered PDF 
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Fig. 4.11. The sample number effect on the PDF/^(x). (a) The data set considered 
is illustrated for A'^ 10^ (b), (c), and (d) show/^Cx) for A'^ 10', A' = lf)\N^ W, 
respectively, where zix = 0. 1 is used. 

is always bigger than the real variance of the data set. The deviation (Ax)^/ 12 can 
be used to assess the accuracy of the filtered PDF. Consequently, the use of the 
filtered PDF for the calculation of probabilities can only be approximately correct 
depending on the relevance of (Ax)^/ 12. For the case that (Ax)^/ 12 is not small, it 
is possible to use the definition 



P{a<X<b) = {9{b-X)-eia-X))-- 



N 



(4.121) 



for the calculation of correct probabilities. The expression A/V^,, / A' follows from 
Eq. (4.1 15) by setting a= x - ISjcI 2 and 6 = x + Ax/ 2: this means A/V^j refers to 
the number of samples in the interval between a and b. 

Sample Number Effect. The calculation of PDFs in dependence on the sample 
number A' is illustrated in Fig. 4.1 1. The data set applied is illustrated for A' = 10' 
in Fig. 4.11a. The data represent normally distributed random numbers with zero 
mean and a standard deviation of one. The PDFs in Figs. 4.1 Ib-d are calculated in 
terms of the definition fj^x) = AN I {N Ax) of a filtered PDF for a fixed interval 
Ax = 0. 1 for varying sample numbers N. The range of x values considered covers 
the range of variations of X values- see Fig. 4.1 la. The positions x at which the 
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Fig. 4.12. The fflter effect on the PDF f^{x). The 
solid lines in (a), (b), (c) showy^(x) for a sample 
number of A'^ 10^ where Ax = (0.1, 0.5, 2) are 
used, respectively. The dashed lines in all the 
three figures showfj^x) for N= 10' and Ax = 0.1 
as a reference. 



PDF is calculated are separated by 0.1. Figure 4.11b reveals the problem of 
calculating a PDF on the basis of a relatively low number A' = 10^ of samples: the 
PDF is significantly affected by noise. An increasing number of samples results in 
a much smoother PDF. The PDF for A' = 10' samples agrees exactly with the 
corresponding normal PDF, which means that a further increase of the sample 
number does not cause any modifications of the PDF. 

Filter Interval Effect. For a relatively low number A' of samples, the need to 
work with relatively smooth PDFs requires the use of a relatively large filter width 
Ax to have a sufficient number of samples in the intervals. Figure 4.12 illustrates 
the effect of Ax variations for PDFs with a number A' = 10^ of samples. The PDF 
with A'^= 10' and zix = 0.1 is given as a reference. These figures illustrate again the 
problem of calculating PDFs from a data set with a limited number of samples. 
For a small interval Ax = 0.1 the calculated PDF is significantly influenced by 
noise due to the low number of samples in the intervals. For a large interval Ax = 2 
the PDF is a relatively smooth function. However, the large filter width Ax applied 
leads to the fact that the PDF is smeared out: the peak value is signifi-cantly 
underpredicted and the probability for high deviations from the mean value is 
overpredicted. There is no unique solution for determining an optimal filter width: 
Ax has to be large enough such that the noise effect is damped out but sufficiently 
small such that the PDF is not smeared out too much. For the case considered, the 
filtered PDF with Ax = 0.5 represents such a reasonable choice. 
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4.4.2 The First Fundamental Theorem of Probability 

The Problem with Randomness. The problem related to the analysis of ran- 
dom numbers, which can be obtained from measurements or models that involve 
randomness, is that every calculation of statistics does provide different results in 
general. For example, the calculation of a mean value 

/'^=^i^,- (4.122) 

based on a finite number A'^ of samples will provide different results depending on 
the number A'^ of sample values applied (and the set of sample values X,, X,, ... X,^ 
used for a given N). It is often the case that there is no way to change this situation 
because there are no other data. However, what we would like to know is under 
which conditions we will have exact results that can be reproduced. This is rele- 
vant to the understanding of the standard of results: it makes a difference to know 
that a particular result is just random, or to know that our result is very close to the 
exact result, or to know that we have an exact result. 

The Law of Large Numbers. After 20 years of analyzing the latter problem, 
Jacob Bernoulli published in 1713 an answer to this relevant question. He named 
his conclusion the "Golden Theorem". In 1835, Simeon Denis Poisson introduced 
the name "The Law of Large Numbers" for Bernoulli's result. Bernoulli's theorem 
states that an infinite sequence of independent and identically distributed random 
numbers Xi, Xj, ... (the random numbers are independent and each variable has 
the same PDF) with finite mean converges to a mean <X> (Ross 2010), 

1 ^ 
lim//^ = lim-2:X,=(X). (4.123) 

The assumption of a finite variance is not necessary. The relevance of Bernoulli's 
theorem is the conclusion that the analysis of random numbers will result in exact 
conclusions that can be reproduced provided the number of samples is huge. 

Illustration. In place of a proof of the Law of Large Numbers, which may be 
found, e.g., in Ross (2010), we consider a numerical illustration of the correctness 
of this law. In particular, we extend this illustration by considering not only the 
mean but also the standard deviation of normally distributed random numbers with 
<X> = 1 and standard deviation <X^ >"" = 1 for a varying number A' of samples. 
The deviations from the exact values <X> = 1 and <X^ >"^ = 1 will be assessed in 
terms of the relative errors 

^---^^ A.,.^A_^. (4.124) 
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Fig. 4.13. (a) The relative error A„, A' "* of the mean value multiplied with the 
square root of the number A'^ of samples. The corresponding relative error A^^A^"^ 
of the standard deviation is shown in (b). 



Here, o^ = <X^ >n'^, where the fluctuation X ^X- /u^ refers to deviations from 
the mean //y obtained for a finite number of samples. The relative errors (4.124) 
multiplied with A'"^ are shown in Fig. 4. 13 in dependence on A'. This figure shows 
that A,„ and A,^ can be described by the relation 

(4.125) 



where ^ is a bounded random variable (Fig. 4.13 suggests that \£ \ < 2). Hence, //^ 
and the standard deviation ov converge to their exact values <X> and <X">"* 
proportional to A' "^. Equation (4.125) can be used to specify the error of using a 
finite number A'^ of samples. For example, we find |A,„| = |AJ < 0.02 and |A„,| = |AJ 
< 0.002 for A^= lO'* and A'^ 10^ respectively. 

Unbiased Estimates. A closely related question is the following one. Usually, 
there is only a finite number of samples that have to be used to find estimates for 
means and variances. Such estimates are still random numbers. A desired property 
of such estimates is that these estimates represent unbiased estimates, this means 
estimates that have mean values that agree with the exact means and variances 
given for an infinite number of samples. The sample mean jUj^ (4.122) represents 
such an unbiased estimate because 



C'HP^)^^ 



X) 



(4.126) 



Here, we replaced <X> by <X> because X^ is just any random realization. An 
unbiased estimate for the variance is given by 



<T N 



1 



N-l~ 






-//a 



(4.127) 



4.4 Data Analysis 147 



This formula differs from the sample mean formula (4.122) by the replacement of 
N by N - 1, which is called the Bessel correction. Bessel's correction is correct 
because there are only A'^ - 1 independent samples available, (Xj - //^, X^- Mm • ■ ^ 
Xfj - //jv), as these samples sum to zero. It is interesting that Eq. (4.127) corrects 
the bias in the estimation of variances, but the corresponding standard deviation 
CTy is biased. A general formula for an unbiased standard deviation does not exist. 



4.4.3 The Second Fundamental Theorem of Probability 

The Generalized Problem with Randomness. The Law of Large numbers 
addresses the question of what is the limiting mean value of a sum Njjp, = Xi + ■ • • 
+ Xf^ of N independent and identically distributed random numbers as A'^ — > co. A 
more general question is the following one: what is the limiting behavior of the 
PDF of a sum N jU,^ ^ Xi + ■■■ + X^j of N independent and identically distributed 
random numbers as A' approaches infinity? An answer to this question is certainly 
helpful. A sum of independent and identically distributed contributions represents 
a reasonable model for random numbers. Thus, the PDF of such random numbers 
may be expected to be a reasonable model for random variables. 

The Central Limit Theorem. A first answer to this question was presented in 
1773 by Abraham de Moivre. In 1812, Pierre-Simon Laplace expanded Moivre's 
findings. The result of Laplace's analysis can be summarized in the following 
way: Let X„X2, ■■■, X^^be a sequence of independent and identically distributed 
random numbers each having a finite mean // and a finite variance cr > 0. Further, 
let Z)^ be the PDF of the sum of A' values X/. Then, the Central Limit Theorem 
says that D^j converges independent of the original PDF to a normal distribution 
with mean N jU and variance A' o^, 

J:X_~D^ ^ \\mD^=^(NM,Na^). (4.128) 

,=1 N^^. 

The symbol 5V(M, V) used here refers to a normal distribution with mean M and 
variance V. Alternatively, the Central Limit Theorem can be also presented for the 
PDF dfi of mean values <X>,^, 



1 ^ 
—T^,~d^ => limaf^=^ 






(4.129) 



These two statements of the Central Limit Theorem are in consistency with the 
conclusions (10.75) and (10.76) for the sum and the mean of normally distributed 
random numbers, respectively. 
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Fig. 4.14. The Central Limit Theorem, (a) The initial uniform PDF. The other fig- 
ures show PDFs of the means of 2, 4, 8, 16, 32 samples (solid lines). Correspond- 
ing normal PDFs are shown in (c), (d), (e), and (f) by dashed lines. 



Illustration. The Central Limit Theorem is illustrated in Figs. 4.14 and 4.15. 
The initial PDFs in these figures show the PDFs of A' = 10* random numbers. The 
PDFs are calculated according tofj^x) = AN/ [A' Ax] with Ax = 0.005. The initial 
PDF in Fig. 4.14 represents a uniform PDF given by Eq. (4.33), where a = and 
Z> = 1. The initial PDF in Fig. 4.15 represents a beta PDF given by (4.107), where 
a^ b ^ 0.3. The following PDFs in these two figures represent the PDFs of mean 
values of 2, 4, 8, 16, 32 random numbers, respectively. The sequences of PDFs in 
these plots describe an increased mixing between the values of random numbers. 
The initial PDFs represent two sorts of unmixed random numbers: the uniform 
PDF defines an equal probability for all states between zero and one, and the beta 
PDF considers only two basic states (one state where the random variable X « 0, 
and another state where the random variable X« 1). The mean values of 2, 4, 8, 
16, 32 random numbers characterize an increased mixing intensity of random 
numbers. Therefore, the corresponding PDFs are characterized by a growing peak 
value and a decreasing variance. These PDF shapes indicate decreasing deviations 
of the mean value <X> = 0.5 with a growing number of samples involved in the 
mean values. In agreement with the Central Limit Theorem, we find independent 
of the initial PDF decreasing deviations between these PDFs and a normal PDF 
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Fig. 4.15. The Central Limit Theorem, (a) The initial beta PDF. The other figures 
show PDFs of the means of 2, 4, 8, 16, 32 samples (solid lines). Corresponding 
normal PDFs are shown in (d), (e), and (f) by dashed lines. 



for an increasing number A' of sample values involved. According to the Central 
Limit Theorem (4.129), the normal PDFs are given by 



/w = 



1 



V^^CTq 



m 



.exp<^ 



(jc-//o)' 

1g^ IN 



(4.130) 



Here, //q ^nd a^ are the mean and variance of the initial PDFs. For the uniform 
initial PDF, we have the values /^ = 0.5 and a^ = \IYl. Regarding the beta initial 
PDF, we have /^ = 0.5 and a^ = 13/32. 

Entropy. The PDF for the mean of eight samples is already very close to a 
normal PDF. The difference to the normal PDF can be quantitatively assessed by 
calculating the entropy according to Eq. (4.53). As shown above, a normal PDF 
has a higher entropy than any other PDF with the same mean and variance. The 
latter fact is illustrated in Fig. 4.16 that shows the entropy S for the PDFs given in 
Figs. 4.14 and 4.15 in comparison to the entropy 5 = ln(2 e n o^)"^ of normal 
PDFs. Also, the entropy of non-normal PDFs is always smaller than the entropy of 
the normal PDFs. These figures also reveal that the non-normal PDFs quickly 
converge to the corresponding normal PDFs. For PDFs for the mean of eight 
samples there is no observable difference between the entropy values. 
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Fig. 4.16. The entropy is shown by a dashed line in (a) with regard to the Fig. 4.14 
PDFs and in (b) with regard to the Fig. 4.15 PDFs. The entropy of the cor- 
responding normal PDFs is given by solid lines. 



4.5 Real Distributions 

Let us consider now the PDFs derived from real data and the suitability of PDF 
modehng concepts discussed above. In particular, we will consider the data of 
velocity components and temperature that were measured by Chu et al. (1996) in 
the atmospheric surface layer for different stability conditions. Such PDFs are of 
interest to our daily life, for example, for the assessment of the probability for 
finding any (extreme) events. Also, such PDFs are used for the evaluation of 
atmospheric turbulence models, and they provide a basic ingredient for the 
development of models for the transport of species in the atmosphere (Luhar et al. 
1996). 



4.5.1 Velocity and Temperature Data 



Atmospheric Stability. Measurements for different atmospheric stabilities (in 
particular for stable, neutral, and unstable conditions) will be considered below. 
Let us briefly describe these conditions to prepare these discussions. Atmospheric 
stability is the degree to which the atmosphere will support, tolerate, or suppress 
vertical motions. In a stable atmosphere, a parcel of air that is displaced upwards 
will tend to return to its original level. In an unstable atmosphere, a parcel of air 
displaced upwards will continue to rise. Stable conditions typically occur during 
nighttime or in winter situations. Unstable conditions typically occur during day- 
time at summer when the Sun is shining. Neutral conditions typically occur in the 
transition from day to night. 
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Table 4.1 Statistical characteristics of the velocity and temperature PDFs. 







<X> 


<X^> 


WJj 


nin 


mj 


'"6 


Stable Case: 


u 


2.89 


0.17 


0.17 


3.02 


2.93 


17.66 




V 


0.00 


0.14 


-0.37 


2.27 


-2.30 


8.54 




w 


0.04 


0.02 


0.19 


5.47 


2.36 


78.44 




T 


21.89 


0.36 


0.09 


2.64 


0.89 


10.96 


Neutral Case: 


u 


6.94 


0.54 


0.08 


3.11 


1.05 


17.49 




V 


0.00 


0.28 


-0.05 


3.41 


-0.43 


21.03 




w 


0.22 


0.13 


0.14 


3.77 


1.83 


29.10 




T 


31.47 


0.23 


0.56 


3.08 


4.65 


17.20 


Unstable Case: 


u 


1.87 


0.44 


-0.01 


2.27 


0.13 


7.12 




V 


0.00 


0.35 


-0.01 


2.86 


0.32 


13.11 




w 


0.01 


0.07 


0.32 


3.13 


2.86 


17.57 




T 


34.80 


0.49 


1.19 


5.10 


16.27 


66.43 



Measurements. Velocity and temperature measurements were performed at a 
height between 2 and 3 m above the surface. For each stability condition, 50,400 
velocity and temperature values were measured over a recording period of 15 min. 
The short sampling period was needed to ensure steady state in the mean meteoro- 
logical conditions. Each data set contains the longitudinal velocity component u, 
lateral velocity component v, vertical velocity component w, and air temperature 
T. The velocities were measured in m / s and the temperature in °C. All the details 
about the velocity and temperature data can be found elsewhere (Chu et al. 1996). 

PDF Calculation. The velocity and temperature PDFs obtained from these data 
are shown in Fig. 4.17. The PDFs were calculated as described in Sect. 4.4.1. 
A relatively small filter interval Ax = 0.2 was used to ensure that the filtering does 
only have a little influence on the calculated PDF. The application of a larger filter 
interval zlx would result, for example, in a reduction of the PDF maximum. The 
PDF values were calculated at sample space values that are separated by 0.02. 
Typical characteristics of these PDFs are given in Table 4.1. Here, <X> and 
< Jf ^ > refer to the mean and variance of the variables considered, respectively. 
The variables m^, m^, m^, and m^ represent normalized standardized central mo- 
ments of third, fourth, fifth, and sixth order. 



X') X') X') X' 



. (4.131) 



The superskewness m^ and superflatness m^ values are included in Table 4.1 in 
addition to the skewness m^ and flatness m^ values because these variables will be 
used below in Sect. 4.5.3. 
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Fig. 4.17. The solid lines show velocity (m, v, w) and temperature (7) PDF data for 
stable {left-hand side), neutral (middle), and unstable (right-hand side) cases. The 
dashed lines show the corresponding normal PDFs. 
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Fig. 4.18. The solid lines show the natural logarithm of velocity (m, v, w) and 
temperature {T) PDF data for stable {left-hand side), neutral (middle) and unstable 
(right-hand side) cases. The dashed lines show In/of the normal PDFs. 
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4.5.2 Normal Probability Density Functions 

Normal Velocity PDFs. According to the Central Limit Theorem, the natural 
way to model PDFs derived from measurements is the use of normal PDFs. The 
mean values and variances required for the calculation of such normal PDFs are 
given in Table 4.1. Figure 4.17 illustrates the performance of normal PDFs. The 
three velocity PDFs can be described in a reasonable way by a normal PDF. The 
best agreement between the normal PDF and measured PDF is given for neutral 
conditions. This finding is not surprising because of the lack of forces that affect 
air parcels: the randomness under neutral conditions can be seen as purely chaotic. 
The biggest deviation between the normal PDF model and the measured PDFs is 
given regarding the u-VDV and u-PDF under stable conditions. Oscillations of air 
parcels about the equilibrium position under stable conditions imply that these 
turbulent motions are not completely random. The existence of such deterministic 
oscillations leads to the appearance of two different modes, see, in particular, the 
bimodal structure of the u-PDF. 

Temperature PDFs. The normal PDF represents a reasonable model for the 
temperature PDF under stable conditions, but the performance of the normal PDF 
under neutral and unstable conditions is not very good. Under unstable conditions, 
the temperature PDF can be considered to represent a superposition of two modes. 
There is a first mode related to relatively warm particles that move up. This mode 
has a relatively large variance and, therefore, a relatively low peak value. There is 
also a second mode related to relatively cold particles that move down. This mode 
has a smaller variance and a higher peak value than the first mode. Mathematical- 
ly, these modes imply a high skewness value m^= 1.19 under unstable conditions. 
A normal PDF, which has a zero skewness value, cannot represent a good model 
for such conditions. It is interesting to note that the temperature PDF under neutral 
conditions also has a significant skewness ofm^ = 0.56. A possible reason for this 
finding is that the conditions were not truly neutral but affected by instability. 

Problem Analysis. Which model can describe the measured PDFs in a better 
way? A look at the non-normal PDF models described in Sect. 4.3 shows that 
these models do not represent alternative choices. It would be possible to use the 
gamma PDF for the modeling of the skewed PDFs, but such a model does not 
represent a general model because it cannot properly describe PDFs that are close 
to a normal PDF. To see how an improved model can be designed, it is helpful to 
consider bi/that is shown in Fig. 4.18. The PDFs can be described by a normal 
PDF if In/ can be approximated by a quadratic function. The comparison with the 
corresponding normal PDFs shows that the latter is not always possible with a 
reasonable accuracy. A way to improve the PDF model performance is to use a 
curve that offers a higher flexibility than a quadratic function, this means a poly- 
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nomial of an order higher than two. The application of a polynomial of third order 
results in a PDF model where the PDF diverges at high positive or negative values 
of the sample space variable (a third-order polynomial will approach infinity for 
large positive or negative values of the sample space variable such that the corre- 
sponding PDF diverges). Therefore, the next better approximation compared to 
the use of a second-order polynomial is the use of a fourth-order polynomial for 
the modeling of Inf. The development of such a PDF model will be described 
next. 



4.5.3 Statistically Most-Likely Probability Density Functions 



Fourth-Order SML PDF. According to the discussion in the preceding para- 
graph we consider a fourth-order SML PDF. This PDF has a maximal entropy S 
(uncertainty) among all PDFs for which the first four moments agree with the first 
four moments of any data set. Equation (4.59) shows that this PDF is given by the 
expression 



fix) = exp 



A„ — X. 



x-{x) xA--{^)y ^A--{^)y ^A--{^)y 



X' 



X' 



X' 



X' 



(4.132) 
The use of this model requires the determination of the five model parameters X^, 
/li, Xi, X^, and X^. There are five conditions for the calculation of these parameters 
given by the constraint that^-^^) integrates to one, and the four conditions that the 
first four moments off{x) agree with the first four moments of any data set. 

Model Parameter Calculation. It is convenient to differentiate /(x) in order to 
simplify the calculation of the model parameters. 
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(4.133) 



By multiplying this relation with < X^ >"' / Xi and writing the term that involves 
the product/(x - <X>) on the left-hand side we obtain 
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(4.134) 
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The muhiplication of this relation with appropriate powers of (x - <X>) / < X^ >"* 
and integration then provides the conditions 

= ^j(x-(X))/Jx = -A±iLlil!!!l, (4.135a) 

f(x-(x))V^x = - ^3^'3+f----\ (4.135b) 



x^Y' ' " ' I 



\^\[. - {X)r fd. = _ A±V!l±A!!!l, (4.135c) 



3 , yr...- .-„.-- ^^ 



— j(x-(x))\/-Jx = -A!!!i±A^!!i±A!!k^, (4.i35d) 



X') ^2 



where m^, m^, m^, and /Wg are defined by Eq. (4.131). Partial integration has to be 
applied to treat terms involving dfldx. The conditions (4.135) can be also written 

1, + /Ij + X^m^ = 0, (4.136a) 

A^ + /IjOTj + X^m^ = 1, (4. 136b) 

A^ + X^m^ + Ajin^ + X^m^ - 0, (4.136c) 

/ijOTj + A2m^ + X^m^ + A^nif^ - 3. (4.136d) 

Equations (4.136a-b) can be used to express /L^ and /I2 in terms of /I3 and /I4, 

A^ = -/I3 - A^m^, X^-X- IjOTj - ?L^m^. (4. 137) 

We use these relations for replacing /I, and /I2 such that (4. 136c-d) read 

-/I3 - /I4/W3 + (1 - /IjOTj - Xami)m-^ + X^m^^ + X^m^ = aA^ + bX^ + m^ = 0, 

—(Xj + X^mj)mj + (1 ~ X^nij — X^m^)m^ + X^m^ + X^nif^ - bX^ + cX^ + in^ =3. 

(4.138) 
Here, the abbreviations a, b, and c are given by the expressions 

a = m^-m^ -1, b - m^ — m^m^ — m^, c - m^ —m^ —m^. (4.139) 



The solutions of the linear equation system (4. 138) are then given by 



The values of X^ and X2 can be obtained by using the latter relations for X^ and X^. 
The parameter Xq that is not determined by these relations can be calculated finally 
by the constraint that the model (4. 132) does satisfy the normalization condition to 
integrate to one. 
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Fig. 4.19. The solid lines show velocity (m, v, w) and temperature (7) PDF data for 
stable {left-hand side), neutral (middle), and unstable (right-hand side) cases. The 
dashed lines show the fourth-order SML PDF model (4.132). 
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Application of tiie Fourth-Order SML PDF Model. The performance of the 
fourth-order SML PDF model (4.132) is shown in Fig. 4.19. The data required to 
determine the model parameters Xy, X^, /I3, /I4 are given in Table 4.1, and X^ was 
calculated such that/(x) integrates to one. Figure 4.19 shows that the fourth-order 
model (4.132) clearly improves the agreement between the modeled PDF and 
measured PDF. The model (4.132) is capable of accounting for the skewness (see 
the (Tb) and (Tc) figures) and the flatness of PDFs (see the (uc) figure). On the 
other hand, Eq. (4.132) is not a perfect model. The model performs better than the 
normal PDF model regarding the (ua) and (va) cases, but the model does not 
represent the bimodal mode structure of these PDFs. A way to improve the model 
performance would be given by extending the fourth-order PDF model to a sixth- 
(or higher) order PDF model. The obvious disadvantage of such an approach is the 
growing complexity of relations that determine the model parameters. 

Comparison with Other PDF Models. The use of a SML PDF model is not 
the only way to address deficiencies of the normal PDF model. For example, a 
superposition of two normal PDFs can be used to model the PDFs shown in 
Fig. 4.17 (Luhar et al. 1996). However, it often turns out that the performance of 
such empirical PDF models is not better than the performance of the model 
(4.132). The significant disadvantage of such other models is related to the 
calculation of model parameters. For example, the PDF model involving two nor- 
mal PDFs requires the numerical solution of two nonlinear equations to determine 
all the model parameters. The solution of these nonlinear equations is not simple: 
there may appear several solutions, or it is possible that there are no solutions. To 
avoid these numerical problems one can simplify these equations on the basis of 
empirical assumptions (Luhar et al. 1996). The disadvantage of this approach is 
that it is not easy to know under which conditions such empirical assumptions are 
applicable. These problems dot not appear if a SML PDF model is used. In this 
case, the model parameters are related analytically to data statistics. 



4,6 Summary 

Let us summarize the basic observations made in this chapter with regard to the 
analysis and modeling (in particular the availability and suitability of modeling 
concepts) of the stochastic state of one random variable. 

Data Analysis. First of all, random data are analyzed in terms of their mean 
value and standard deviation. These quantities provide a basic understanding of 
the value and amount of randomness that we can expect. We are also interested in 
PDFs, which give us the possibility to calculate probabilities for certain events. A 
relevant feature of almost all real data is that there are usually a limited number of 
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observations. Such a limited number of samples has significant consequences for 
the calculation of mean values, standard deviations and PDFs: it means that these 
quantities do represent again random variables. PDFs are often also affected by 
the filtering applied to produce relatively smooth PDF shapes. In order to deal 
with this problem we need a basis that allows us to determine the standard of our 
results (we need to know whether a result obtained is just random, whether the 
result is close to the exact result, or whether we have an exact result). This basis is 
given by the first fundamental theorem of probability. This theorem allows us to 
determine the error due to a finite number of observations: we derived the relation 
|A„,| = jAjrfl = \£ I A'""" for the relative errors of means and variances. A correspond- 
ing relation applies to PDFs, which do also represent a mean. Unfortunately, it 
needs huge numbers of samples to derive conclusions that are independent of the 
sample number. For example, we needs 10* (or 10*) sample values to have an error 
ofabout2%(or0.2%). 

Modeling Concepts. Which theoretical basis can we use to determine PDFs? 
There is only one general concept given by a SML PDF of a certain order. Such 
PDFs can be constructed such that any even number of PDF moments agrees with 
the same even number of moments of any data set. A normal PDF represents a 
second-order SML PDF, this means a normal PDF is the best PDF choice among 
all PDFs that agree with a given mean and standard deviation of data. Also, the 
second fundamental theorem of probability supports the use of a normal PDF 
under conditions where we do only have randomness (i.e., conditions where we do 
not have any deterministic processes in addition to randomness). For such 
conditions there is no need to use, for example, a fourth-order SML PDF model 
that accounts (in terms of the consideration of skewness and flatness) for deter- 
ministic processes in addition to randomness (see the discussion in Sect. 4.5). It is 
relevant to note that the concept of SML PDFs can be used for both unbounded 
and bounded variables (Heinz 2003). However, applications of SML PDFs to the 
case of bounded variables lead to nontrivial problems regarding the calculation of 
model parameters. For such cases it is often more convenient to use other PDFs, 
as given by gamma PDFs for non-negative random variables and beta PDFs for 
bounded variables. The advantage of gamma and beta PDFs is the simple relation 
between model parameters and means and variances. However, such PDFs do not 
have any theoretical support - they do only represent empirical assumptions. A 
significant disadvantage of such empirical concepts is that the extension of such 
concepts to the case of several variables is rather complicated due to the limited 
possibilities to account for correlations of random variables. The application of the 
SML PDF concept is also beneficial with regard to the latter question because 
SML PDFs can be constructed for several variables. 
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Applicability of Modeling Concepts. How useful are theoretical concepts for 
the modeling of PDFs? This question was addressed in terms of the discussion of 
the modeling of atmospheric velocity and temperature PDFs. It turns out that the 
normal PDF model represents a very valuable model: many PDFs can be accurate- 
ly represented, and the normal PDF is at least a reasonable model in general. In 
particular, a normal PDF model is a very good model as long as only randomness 
is involved (as long as completely unorganized chaotic motions are considered). 
The normal PDF concept becomes questionable if deterministic trends are also 
involved (for example, oscillations of air parcels under stable conditions or 
upward and downward motions of warm and cold air parcels under unstable 
conditions). A way to improve the performance of models under conditions that 
involve deterministic trends is to develop SML PDFs of higher order, which 
usually improves the modeling performance. A disadvantage of such higher-order 
PDFs is that they need more information (e.g., knowledge about third, fourth, fifth 
and sixth-order central moments), and the complexity of model parameter calcula- 
tions increases. 



4,7 Exercises 



4.2.1 Consider/(?) = exp(-|?-?o|/ r)/(2 r) with rand Iq as any model parameters. 

a) Which properties of J{t) are required conditions for the conclusion that 
/{t) approaches a delta function, /(^ — > (5(?-?o), for r— > 0? 

b) Show analytically that J{t) -^ S{t-t^) for t^>-Q. 

A.1.1 Consider the uniform PDF given by Eq. (4.33). 

a) Is it possible that the uniform PDF (4.33) behaves like a delta function? 
Explain the condition if the answer is yes. 

b) If the answer is yes, is there (apart from the different variables applied) 
any difference between the delta function provided by the uniform PDF 
and the delta function provided hyf{t) considered in exercise 4.2.1? 

4.2.3 Show the correctness of the relation 

gi^) — —dx = - -—(x^)5{x - Xg)dx = — -(xo), 

ax ax ax 

where g(x) is any given function of x. 

4.2.4 A random numberXis uniformly distributed over the interval (-2, 3). 

a) Calculate the probability P{-3 <X< 0). 

b) Calculate the probability P{0<X). 
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4.2.5 Consider the following problem (see Ross 2010). Buses arrive at a certain 
stop every 15 min starting at 7 a.m. A passenger arrives at the stop at a 
time that is uniformly distributed between 7:00 a.m. and 7:30 a.m. (X refers 
to the number of minutes past 7 a.m., and X is uniformly distributed over 
the interval (0, 30)). 

a) Find the probability that he waits less than 2 min for a bus. 

b) Find the probability that he waits more than 10 min for a bus. 

4.3.1 Consider the normal PDF (4.72) and the uniform PDF (4.33). 

a) Calculate the entropy of the normal PDF. 

b) Calculate the entropy of the uniform PDF. 

c) Which PDF has the higher entropy if the normal PDF and uniform PDF 
have the same mean and variance? 

d) Consider a normal PDF and uniform PDF that have the same mean and 
variance. The uniform PDF seems to represent the SML PDF: this PDF 
treats the probability for all possible states in the same way, there is no 
preference of certain states. Why is the normal PDF and not the uniform 
PDF the SML PDF? 

4.3.2 The intelligence quotient (IQ) is seen to be normally distributed with mean 
100 and standard deviation 15. 

a) What is the probability that the IQ of a randomly selected person will be 
between 80 and 120? 

b) What is the probability that this IQ will be above 150? 

4.3.3 The heights of adult males in the United States are considered to be nor- 
mally distributed with mean 1.75 m and standard deviation 0.07 m. 

a) Calculate the probability that a randomly chosen adult male is between 
1.5 m and 1.7 m tall. 

b) What percentage of the adult male population is more than 1.95 m tall? 

4.3.4 The length of human gestation is seen to be normally distributed with mean 
268 days and standard deviation 15 days. 

a) What is the probability that a pregnancy takes 268 ± 3 days? 

b) Consider this problem (see Ross 2010). The defendant in a suit is able to 
prove that he was out of the country during a period that began 290 days 
before the birth of the child and ended 240 days before the birth. If the 
defendant was the father of the child, what is the probability that the 
mother could have had the very long or very short gestation? 

4.3.5 The PDF fix) of a non-negative random variable Xhas the structure 
f{x) = exp[ a - b X - c x^ J, 
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where a, b, and c are any model parameters. It is assumed that c > so that 
f{x) does not diverge for large x values. 

a) Calculate the parameter a in terms of the normalization condition. 

b) Show that the mean value <X> implied by the PDF/(x) is given by 

^ ' 2c 

c) Show that the variance implied by this PDF is given by the expression 

l~,\ l-b(X) , ,2 

Hint: use the approach applied to the analysis of the normal PDF in 
Sect. 4.3.2 for doing this (see the second paragraph of this section). 

4.3.6 Consider again the PDFy(x) given in exercise 4.3.5. 

a) Specify the results a), b), and c) of exercise 4.3.5 for the case that c = 0. 
Hint; the simplest approach is to consider the PDF for c = directly. 

b) Explain the consequences of your results. 

4.3.7 Consider the PDF/(x) and its consequences given in exercise 4.3.5. 

a) Compare the PDF with other PDFs discussed in this chapter: which kind 
of PDF is considered? 

b) Explain how the PDF parameters can be calculated so that the PDFy(x) 
agrees with any given mean /.t and any given variance d^. 

4.3.8 Consider the PDF/(x) and its consequences given in exercise 4.3.5. 

a) Develop a numerical scheme for finding the PDF parameters a, b, and c 
so that the PDF agrees with any given mean // and given variance cP'. 

b) Graph the PDF for the cases that a= 0.25, a= 0.5, and (7=1, where 
// = 1 for all the three cases. Report the b and c values for these cases. 

c) Discuss the three PDFs obtained in b) in comparison to the PDF shapes 
shown for the gamma PDF in Fig. 4.9. 

4.3.9 Consider the gamma PDF discussed in Sect. 4.3.3. 

a) Calculate the skewness m^ and flatness m^ for the gamma function. 

b) Under which condition are the skewness and flatness values equal to the 
values wjj = and m^^ = 3 of a normal PDF? 

c) What does this condition mean regarding the mean value and variance? 

d) Which PDF shape is related to this condition? 

4.3.10 Consider the beta PDF discussed in Sect. 4.3.4. 

a) Show that the moments of the beta PDF (^=1,2, . . .) are given by 

,,,\ _ {a + k-\){a + k-2)---a 



X' 



{a+b + k- \){a + b + k-2)---{a+b) 
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b) Use these moment relations to show that the PDF parameters a and b are 
related to the mean and variance via the relations 



a= X' 



W(l-(^)) 



X- 



. = (1- 



X 



(^)(i-W) 



4.3.11 Consider the beta PDF discussed in Sect. 4.3.4. The skewness m^ and the 
flatness m^ of the beta function are given by the expressions 

2ib-ay ^ 



a+b + 2 



a+b + l 



■b + l 



ab 



a+b + 2) 



1- 



ab{a +b + 2) 



a) Under which condition are the skewness and flatness values equal to the 
values m^ = and m^ = 3 of a normal PDF? 

b) What does this condition mean regarding the mean value and variance? 

c) Which PDF shape is related to this condition? 

4.3.12 Consider the following problem (Stewart 2006). The hydrogen atom con- 
sists of one proton in the nucleus and one electron, which moves about the 
nucleus. The electron does not move in a well-defined orbit, but there is a 
probability for finding the electron at a certain distance from the nucleus. 
The PDF is given by p{r) = 4 /-^ exp(-2 r / ag) / a^ for r > 0, where Aq " 
5.59 X 10 " m is the Bohr radius. The integral over this PDF from zero to r 
gives the probability that the electron will be found within a sphere of 
radius r meters centered at the nucleus. 

a) Calculate the mean distance R of the electron from the nucleus. 

b) Calculate the variance a^ of the PDF. 

c) Show that the PDF is a gamma PDF. Hint: use T{n +\) = n\ for integer 
values of «. 

d) For what value of r does /?(/') have its maximum value? 

e) Find the probability that the electron will be within the sphere of radius 
3 flo centered at the nucleus. 

Hint: the following integrals will be helpful (a is any constant). 



. 3 3 

\a X 



-3a X + 6ax -6j, 



\x e''^dx- — j-\a x -lax + 2), jxe^'dx- — ^{a 

ax . . 

\x e''^dx- — Y^f X - 4a X +I2a x -24ax + 24]. 
4.4.1 The variance o^Mea,, of the mean jU^j ofN independent samples is defined by 

CT\...={[M.-{X)Y)J\J^P.-{X) 
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a) Show that o^uea,, ^ o^lN, where a^ = <{X- <X>f>. 

b) Relate oi^,„„ = cr/A^"- obtained here to the finding |A„J = \AJ ^\s\IN^'" 
obtained in Sect. 4.4.2. 

4.4.2 Consider the variance cr^v of A'^ independent samples, which is defined by 

<^^N =——-ux.-mnY- 

a) Calculate the mean value <c^if' to show that this expression represents 
an unbiased estimate for the variance (to show that <crp/> = ct^, where cr 
= <{X- <X>f>). Hint: Replace X, - /u^ in cr^ by X^ - <X> - (ju^j - <X>). 
Distribute the square. Use the definitions of cr and o^Mea,, " c/^N. 

b) How relevant is Bessel's correction (the use of N - 1 instead of A'^in the 
variance formula)? 

4.4.3 Consider the case that there are insufficient data to estimate a meaningful 
variance. What is the best possible characterization of any random variable 
Xfor this case? Hint: consider the mean squared error g(x) = <(X-x)'> of 
any estimate x of X and calculate the minimum of g(x). 

4.5.1 Consider Eqs. (4.136) for the calculation of the model parameters of the 
fourth-order SML PDF model (see Sect. 4.5.3). 

a) Use Eqs. (4.136) for calculating the model parameters of a second-order 
SML PDF model. 

b) For which data set do these relations imply that the fourth-order SML 
PDF reduces to a second-order SML PDF model? 

4.5.2 Consider the fourth-order SML PDF model discussed in Sect. 4.5.3. The 
relations (4.136) represent four equations for the four unknowns /li, /Ij, /I3, 
and /I4. According to the model development, 2,, A2, /I3, and A^ can be 
calculated by knowing the mean, the variance, m^ and m^. 

a) Explain how the parameters m^ and mg are mathematically determined in 
terms of Eqs. (4.136). 

b) Explain the use of Eqs. (4.136) for the calculation of a fourth-order SML 
PDF without knowing anything about m^ and m^. 

4.5.3 Consider the temperature anomaly data presented in Table 1.7. 

a) Consider the data from 1850-1930. Calculate the filtered PDF for these 
data for three choices of the filter interval AT. Which AT represents the 
best choice? 

b) Consider the data from 1930-2008. Calculate the filtered PDF for these 
data by using the best available Ar determined in a). 

c) Use the comparison of these two PDFs to predict the potential features 
of the PDF for the next eighty years. 
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The models discussed in the previous chapters were based on observations: the 
goal was to develop models that agree with any given observations. A different 
modeling approach is the use of mathematical concepts that explain changes of 
variables (e.g., by means of a formula that provides the change of any population 
over a certain time). The comparison of predictions implied by such concepts with 
observations is usually needed to show the validity of the ideas applied, but the 
model development can be performed without the use of observations. Usually, 
such model developments are performed on the basis of difference equations or 
differential equations. Differential equations are applied for postulating general 
laws (e.g., the laws of mechanics or the laws of population ecology). Characteris- 
tic features of such equations for the deterministic evolution of single variables 
and several variables will be described in Chaps. 7 and 9, respectively. Difference 
equations are useful for the modeling of processes that require the modeling of 
changes over a specific time period (e.g., the change of any population per year). 
Apart from that, difference equations provide the basis for the mathematical 
formulation, numerical solution, and in some cases even the analytical solution of 
differential equations. Therefore, characteristic features of deterministic difference 
equations will be discussed in this chapter in order to prepare the discussion of 
differential equations in Chaps. 7 and 9. 

Section 5.1 explains the motivation for this modeling approach. The simplest 
way to use difference equations for the modeling of deterministic changes (linear 
first-order difference equations) will be discussed in Sect. 5.2. Sections 5.3 and 
5.4 describe extensions of such equations: characteristic features of second-order 
linear equations and first-order nonlinear difference equations will be discussed. 
Relations between difference and differential equations and the basic advantages 
and disadvantages of these equations will be described in Sect. 5.5. The obser- 
vations on difference equation models will be summarized in Sect. 5.6. 
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5.1 Motivation 

National Income. Let us discuss an example to illustrate the benefits of using 
difference equations for the modeling of changes. The national income represents 
a measure for the total level of economic activity within a country during a certain 
period of time. The economic activity of a country determines the living standard 
of people: a recession decreases and a boom increases the hving standard. Thus, 
there is a great interest in the variations of the national income to understand, for 
example, the effect of factors that influence the national income. There is a variety 
of possibilities to define the national income and its influence factors. Here, we 
will focus on a relatively simple model pioneered by the American economist 
P. A. Samuelson (1939). Samuelson's model describes the national income j,, of a 
country during an accounting period « = 0, 1,2, ... (n may refer to one year) as a 
sum of three contributions, 

j„=C„+/„+G,, (5.1) 

C„ is the amount spent on consumer goods (food, housing, clothing, . . .) during the 
accounting period. /„ is the induced private investment (the amount invested in 
new machinery, equipment, . . .). G„ is the amount spent by the government (social 
security, health, education, infrastructure investments, research spending, ...). To 
calculate y„ it is needed to relate C„, /„, and G„ to y„. The consumption expenditure 
C„ may be expected to be proportional to the national income of the preceding 
period (an increase in the living standard leads to an increase in consumption), 

C„=ay„_^. (5.2) 

Here, the constant of proportionality < « < 1 is called the marginal propensity to 
consume. The induced private investment /„ can be modeled in several ways. One 
possible assumption (which is called the acceleration principle) is that /„ is propor- 
tional to the increase in consumption of the period considered over the preceding 
period (an increasing living standard over the previous period leads to increased 
investments to offer additional goods), 

/„=/?(C„-C„_,) = «/?(j„_,-j„_,), (5.3) 

where the model parameter /? > 0. Samuelson considered variations < /? < 4. 
Finally, a simple model for the government expenditure G„ is that G„ is constant, 

G„ = r- (5.4) 

By using appropriate units we may assume that 7=1. According to the three 
assumptions (5.2)-(5.4), Samuelson's national income model reads 

y„ = « j„-i + « /? iy„-i - yn-2 ) + r = « (i + /?) y„-i -aP y„-i + r- (5.5) 
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Starting with any initial value y^, the model (5.5) can be used to calculate the 
evolution of the national income in dependence on jg and the model parameters a, 
P, and J. This model is capable of describing a wide range of scenarios. A detailed 
analysis of features of this model will be provided in Sect. 5.3. 

Questions About Difference Equation Models. The national income problem 
considered in the preceding paragraph (or any other problem involving difference 
equations) leads to several questions related to the use of such equations for the 
modeling of problems: 

• Which processes can be described by which type of difference equation? 

• How can we solve a certain difference equation analytically? 

• How is it possible to adjust model parameters to given observations? 

As mentioned in the beginning, there is a close relationship between difference 
and differential equations. With regard to these relations, there are questions like: 

• How are difference and differential equations related to each other? 

• What is the advantage and disadvantage of using difference equation models? 

• What is the advantage and disadvantage of using differential equation models? 
These and other questions will be addressed in the following by analyzing the 
features of difference equations and illustrating the benefits of such models by 
means of applications. 



5.2 Linear Changes 

Let us discuss first the simplest concept for a linear difference equation: a first- 
order linear difference equation for y„. We assume that y„ has a given initial value 
Jo at n = 0. The y„ values for «= 1, 2, ... are determined by the equation 

J„=«J„-i+^- (5-6) 

Here, a and h are any model parameters. This equation can be written y^ - y„_i = 
{a - 1) y„_i +b, i.e., Eq. (5.6) provides a formula for changes of %. Equation (5.6) is 
a linear equation because y„ does only appear linearly. This equation is called a 
first-order equation because only y„ and>'„_i are involved (i.e., there are no terms 
y„_2, or j„_3, . . .). Equation (5.6) is called a homogeneous equation i{b = 0. 



5.2.1 Linear First-Order Difference Equations 

Solution. Let us try to find the solution to Eq. (5.6). The solution is a formula 
that gives us the value of;',, based on the knowledge of the parameters a, b, and jg 
involved. To find the solution we have to understand how the value of y„ is 
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affected by the initial value yo. To see this we consider j^„ for « = 1,2, . . ., 

yi^ay^^+b, (5.7a) 

y^^ayi+b, (5.7b) 

y^=ay2+h. (5.7c) 

To relate y„ to yo, we use yi in the relation iory2, and jj in the expression iory^, 

y^=ay^+b, (5.8a) 

y^ =a{ay^+b) + h = a^y^+b{a + \), (5.8b) 

jj = a(a(a Jo +fe) + Z7) + Z' = a^y^ +b(a^ + a + 1). (5.8c) 

According to these expressions, it is reasonable to expect that the general solution 
has the form (n = 0, 1, . . .), 

y„ ^a"y,+b(a"-' +a"-'+--- + a'+a°). (5.9) 

Solution Rewriting. The sum of powers of a represents a geometric series, 

1 " 

a +a H V a +a = . (5.1U) 

\ — a 

The validity of Eq. (5.10) can be seen by introducing the partial sum 

/j-l . n-2 , .1.0 /■ c 1 1 \ 

p,^-a +a -\ \- a + a . (5.11) 

The multiplication of this relation with a results in 

ap,, =a" +a"-'+a"-^+--- + a\ (5.12) 

By taking the difference p„ - a /?„ we find that all terms cancel with the exception 
of two terms: 

P„-ap„='i--a". (5.13) 

The latter relation then provides 

1 " 

P,.=- , (5.14) 

I- a 

which shows the validity of Eq. (5.10). The combination of Eqs. (5.9) and (5.10) 
enables us to write the solution in the following way (« = 0, 1, . . .), 

y„-a"y,+b^-^ = -^ + a"(y,--^] (5.15) 

1-a 1-fl ly I- a J 

The consistency of this solution at a = 1 can be seen by using Eq. (5.9), or by 
using I'Hospital's rule. In both ways we find for a = 1 that 

y„=yo+nb. (5.16) 
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Solution Proof. To prove the correctness of our assumption that the solution y„ 
is given by Eq. (5.15) it is needed to show first that this solution is in agreement 
with the initial value, and second that this solution satisfies the difference equa- 
tion (5.6). By setting « = on both sides of Eq. (5.15) we find 

which means that the solution is in agreement with the initial value. Evidence that 
the solution satisfies the difference equation can be obtained by using Eq. (5.15) in 
the difference equation (5.6), 



b J b 

+ « Jo--, | = « 

I- a \ I- a 



- — + a \yo-- — 

1-fl I I- a 



..b\l + ^\ + a"\ y,--^^\ = -^^ + a"\y. 



+ b 

(5.18) 



I- a J l^' I- a J I- a y I- a 

The rewriting of the last expression in the first line results in the same expression 
on both sides. Thus, that the solution (5.15) solves the difference equation (5.6). 

Solution Features. Characteristic features of solutions to the first-order linear 
difference equation (5.6) are illustrated in Fig. 5.1. Fixed values Z> = 1 andj'o ^0.1 
are applied, and a is given by the values shown in Fig. 5.1. These figures show 
that the value of j„ becomes constant for large « if |fl| < 1. The equilibrium value s 
is given by 

b 



1- 



(5.19) 



as may be seen from Eq. (5.15). The value a = 1 implies a linear function accord- 
ing to Eq. (5.16), and values a > 1 lead to increasing functions. Fluctuations of y^ 
appear if the values of a are negative. The fluctuations are damped out, or they 
have a constant or an increasing amplitude. With regard to applications, it turns 
out that models that involve such deterministic fluctuations are not often helpful - 
it is usually hard to adjust such models to data. 



5.2.2 Interest 

Interest. Let us consider some applications of first-order difference equations 
to finance problems (Fulford et al. 1997) to illustrate the benefits of using such 
equations. A sum of money that is lent to a bank earns interest. Let us denote the 
amount on deposit after n time periods by S„. The interest is denoted by /„. The 
relation between S„ and /„ is given by 

S„=S „_,+!„_,. (5.20) 
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Fig. 5.1. An illustration of features of solutions of the first-order linear difference 
equation (5.6) in dependence on n, where b ^ I and jo "= 0.1. The values of the 
model parameter a are shown in the plots. 

This formula states that the amount on deposit after n periods is equal to the sum 
of the amount on deposit after the previous period plus the interest earned. The 
question now is how the interest /„ is related to S„_i. 

Simple Interest. A simple way of offering interest is given by the option that 
the interest is proportional to the initial sum Sq deposited, called the principal, 

I„-^=pSo- (5.21) 

Here, p is the constant interest rate (for example, p = 0.05 means that there is an 
interest rate of 5%). The use of this expression in (5.20) results in the following 
first-order difference equation for 5„ (« = 0, 1, . . .), 

S„^S„_,+pS,. (5.22) 

This equation represents a specific case of Eq. (5.6) where a = 1 and b ^ p Sq. By 
using the solution (5.16) for fl = 1, the solution of Eq. (5.22) is given by 

S„=SQ+npS,=(l + np)S,. (5.23) 

This formula is called the simple interest formula. According to this option, the 
amount on deposit after n periods is a linear function of the principal Sq. It is 
assumed that n refers to the number of years. 
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Example. An example for the use of the simple interest option is the following 
one. We are interested to know the number k of years needed to double any initial 
interest S^. According to (5.23), the condition for the calculation ofk is given by 

2S,={\ + kp)S,. (5.24) 

Therefore, the number k of years required is A: = 1 1 p. For example, 50 years are 
required to double any principal if the interest rate is/? = 0.02. 

Compound Interest. Another type of interest is given if interest is added to the 
principal at regular intervals, which are called conversion periods. For this case, 
the interest formula is given by 

A,-i=-5„_,. (5.25) 



Here, r refers to the number of conversion periods per year, this means r = \2 
corresponds to a conversion period of one month, and r = 4 corresponds to a 
conversion period of one quarter. For this interest option, the difference equation 
for S„ is given by (« = 0, 1, . . .), 



r 



(5.26) 



This equation is an homogeneous first-order linear difference equation. To use the 
solution formula (5.15), it is required to relate the model parameters/? and r to the 
parameters of the general difference equation (5.6). This comparison shows that 
a = 1 + pi r and 6 = 0. Using a and b from Eq. (5.15) in Eq. (5.26) yields 



(5.27) 



This formula is called the compound interest formula. Regarding the use of this 
formula it is relevant to note that n refers to the number of conversion periods 
(rather than the number of years in the simple interest formula (5.23)). For 
example, to find the amount of deposit after 10 years at compound interest with a 
monthly conversion period, one has to apply n = 120 in Eq. (5.27). 

Effect of Conversion Period. What will be the effect of different conversion 
periods? To address this question, let us consider the condition for the validity of 



\'i" 



1 + 



P 



■y) 



> 1 



P_ 



(5.28) 



which compares the effect of two conversion periods on the interest. We take the 
natural logarithm and divide both sides by np. This leads to 



/ 



^In 
P 



1 + - 



V 



-In 



'ly 



1-I-- 



(5.29) 
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Fig. 5.2. The function/(z) according to Eq. (5.30). 
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By introducing the function 

/(z) = ^^^^^, 

z 

Eq. (5.29) can be also written 

fiz,)>fiZ2l 



(5.30) 



(5.31) 



where Zi ^ p/ I'l and Zj =/?/r2- Figure 5.2 shows that/(z) is a decreasing function 
of z. Therefore, the condition for the validity of Eq. (5.28) is that z, < Zj, which 
requires that r^ > r^- Thus, the higher the number of conversion periods, the higher 
will be the interest made over the same time at a constant interest rate p. 

Comparison of Options. Let us compare the compound interest and the simple 
interest options. This comparison of options can be performed by making use of 
Bernoulli's inequality 

(l + z)">l + «z, (5.32) 

where z > and the integer n > 0. The validity of Bernoulli's inequality can be 
seen by using the binomial theorem for the left-hand side, 

(l + ^)"-Z ,,/' ,^ z'-l + «z + zl„, (5.33) 

k=o k\{n-ky. 

where A„ is a non-negative contribution. We may setz=p/r and replace n by r n 
in Bernoulli's inequality, 



P \ , P , 

' > 1 + /-«— = l + np. 



(5.34) 



Here, n refers to the number of years considered. The right-hand side multiplied 
with Sq gives the amount on deposit made with the simple interest option. The left- 
hand side multiplied with S^ gives the amount on deposit made over the same time 
(over r n conversion periods) with the compound interest option. Therefore, the 
compound interest option produces over the same amount of time at least as much 
interest as the simple interest option if 5o andp are the same. 
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Example. To illustrate the use of the compound interest formula we consider 
the same example as before: we calculate the number k of years needed to double 
any interest S^. According to Eq. (5.27), the condition for k is now given by 

25o=fl + ^J5o. (5.35) 

By taking the natural logarithm of both sides, we find the number k of years to be 
determined by the relation 

ln2 = A:ln[l + ^|. (5.36) 

Thus, it takes 416.2 conversion periods (34.7 years) to double any principal if the 
interest rate is /» = 0.02 and the interest is compounded monthly. The compound 
interest produces the same amount on deposit as the simple interest in less time, 
which agrees with the conclusion obtained in the preceding paragraph. 



5.2.3 Loan Repayments 

Loan Repayments. Loan repayments can be studied in a corresponding way. 
Let us consider the repayment of a housing loan that required any initial debt D^ 
(the amount borrowed from a financial institution). The debt changes due to (/) 
repayments at regular intervals made to reduce the debt, and (//) interest that has to 
be paid on the amount still owing. By considering a compound interest, the debt 
D„ after n payments is given by the following formula 

^"=(1 + 7]^,-.-^- (5.37) 

Here, D„ and D„_i are the debt after n and («-l) payments, respectively, and R is 
the constant repayment. The term involving p I r refers to the compound interest 
option. To enable the use of the general solution formula (5.15) it is needed to 
relate the parameters of the model (5.37) to the parameters a and b in the general 
difference equation (5.6). This comparison reveals that a= 1 + p I r and b = -R. 
According to Eq. (5.15), the solution 5„ of Eq. (5.37) is, therefore, given by 



-■-'-^•-«-^^|^<-fJ--T 



1+^ 



(5.38) 



This formula is called the loan repayment formula. In correspondence to the com- 
pound interest formula (5.27), n refers to the number of conversion periods. 
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Example. To illustrate the use of Eq. (5.38), let us find first the repayment R 
needed to keep the debt equal to the initial debt D^. By setting Z)„ = D^, Eq. (5.38) 
can be written 



A 



1-1 + 



rR 
P 



1-1 + 



(5.39) 



The bracket terms are nonzero for nonzero p and r and « > 0. Hence, this relation 
implies R= pD^,! r. Let us assume now that a monthly repayment of 



R = q^D, 
r 



(5.40) 



is made to reduce the debt, where ^ > 1 . The number k of conversion periods re- 
quired to repay the debt can be calculated for this case on the basis of Eq. (5.38). 
By using R = qpD(,lr and Z)o = in Eq. (5.38), we obtain for k the condition 



= 11 + ^1 D,+qD, 



1-1 + 



r 



■.DAq-{q-\)\\ + 



The corresponding condition for k is given by 

n-l 



k = \n\ 



q 



q-\ 



M\ + - 



r 



(5.41) 



(5.42) 



The total payment Tmade over k conversion periods is k times R = qpO^/r, i.e.. 



qpD^ 



In 



1 



Inl + i^ 



(5.43) 



For/? / r « 1 (as given for example for the case that the interest rate is p ^ 0.02 
and the interest is compounded monthly), \n{l+ p/ r) ^p/ r represents a very good 
approximation. For this case, Eq. (5.43) can be written 



T = qln 



q 

q-\ 



A 



(5.44) 
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The variation of T standardized by Do as function of q is shown in Fig. 5.3. It may 
be seen that monthly payments of ^ = IpD^I r are a requirement to ensure that the 
total payment is below 1.4 Dq. It is interesting to observe that higher payments 
than R = 2pDf,l r have a rather limited effect on the total costs. 



5.3 Linear Changes with Delays 

The second type of difference equation considered is a second-order linear dif- 
ference equation. Initial values yo at « = and yi at « = 1 are considered to be 
given. The y„ values for n = 2, 3, ... are determined by the equation 

y„=Ay^_,+By„_,+C, (5.45) 

where A, B, and C are any model parameters. By adding - y„_i on both sides, this 
equation can be also written as an equation for changes y„ - y„_y Equation (5.45) 
is called homogeneous if C = 0. Equation (5.45) is linear because there are only 
terms that are linear in the function considered. Equation (5.45) is a second-order 
difference equation due to the inclusion of >'„_2 in addition to y„ and j„_i. The con- 
sideration of j„_2 is the reason for talking about the inclusion of delay effects. The 
concept of a first-order difference equation is that the current state y„_^ determines 
the future state j„. A second-order difference equation extends the concept of a 
first-order equation by accounting also for an effect of the previous state j„_2 on 
the future state y„. The latter effect can be seen to be delayed. 



53.1 Linear Second-Order Difference Equations 

Solution Approach. It turns out that the simple approach used to solve a first- 
order linear equation is inappropriate to find a solution for the second-order linear 
equation (5.45). One may expect that the solution will involve two contributions: 
one equilibrium solution (denoted by s), and one transitional solution (denoted by 
z„) that accounts for the transition to the equilibrium. Hence, we write 

>'„=5 + z„. (5.46) 

The equilibrium solution s can be found by replacing j',, by s in Eq. (5.45), 

s = As + Bs + C. (5.47) 

This relation determines the equilibrium solution as 

s = . (5.48) 

l-A-B 
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It is worth noting that such a constant equilibrium solution does not always exist: 
see exercise 5.3.3. The z„ equation can be found by using j„ = ^ + z„ in Eq. (5.45), 

^ - + ',.-4 , ': „ +^„-il + gf . ^ ^ +^„-2] + C, (5.49) 



l-A-B yi-A-B J yi-A-B 

where s ^ C/(l -A-B) is applied. This equation can be written 

z„ - Az„_, - Bz„_, = ^_^_^ (l -A-B-l + A + B) = 0. (5.50) 

Hence, the equation for the transitional solution z„ reads 

z„^Az„_,+Bz^_,. (5.51) 

This is an homogeneous equation as there is no term independent of the function 
z„ for which we solve. To determine the structure of solutions to Eq. (5.51), let us 
consider solutions for the case that A ^ and B = 0, respectively. For i? = we 
find the solution z„ = A" Zq, which corresponds to the solution of the linear first- 
order equation (5.6) for the case that fo = 0. For the case ^ = we find the solution 
z„ = ± B"'^ Zq. The solution for the latter case can be found by assuming a similar 
form of the solution as given for the B ^ case. The validity of z„ = A" Zq and z„ = 
±B"'^Zq can be seen by using these expressions in (5.51) for the cases considered. 
The solutions obtained lead to two conclusions. First, there are two solutions of 
Eq. (5.51) in general. Second, we may assume that each solution consists of a 
constant multiplied with another raised to the power n. Therefore, solutions to the 
homogeneous equation (5.51) can be expected to have the structure 

z„ =Ci'V' +C2'2". (5-52) 

where Ci, C2, ri, and Tj are any constants that have to be specified. According to 
y„ = s + z„, the general solution to Eq. (5.45) is expected to be given by 

y„ = s + Ci r" + Cj r,". (5.53) 

To demonstrate the validity of this assumption we have to show first that this 
expression satisfies the difference equation (5.45). This demonstration will show 
how Ti and rj are related to the model parameters A and B. Second, we have to 
explain under which condition this expression agrees with the initial data. The 
latter will show how Ci and Cj are related to y^ and j',. 

Consistency with Difference Equation. To prove the suitability of Eq. (5.53), 
we have to show that this expression can satisfy the difference equation (5.45). 
According to the discussion in the preceding paragraph, the latter requires 
evidence that the assumption (5.52) for z„ can satisfy the z„ equation (5.51), i.e., 
we have to show that the following relation can be satisfied, 

■;' =A[c,rr' +c,rr']+B[c,rr' +c,rr'). (5.54) 
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The latter condition can be also written 

'(r,'-^r, -5)+C2r2'"'(r/ -^rj -b)=0. (5.55) 



n-2 I 

c, r, 



The terms c^ r," ^ and Ci r" ^ have to be nonzero to enable the calculation of non- 
trivial solutions. Thus, Eq. (5.55) requires that the bracket terms are zero, 

r,'-^r, -fi = 0, r^^ -Ar^-B = (}. (5.56) 

The conditions (5.56) represent the same quadratic equation for the two roots I'l 
and Vj. Thus, ri and rj can be calculated as roots of the equation 

r^-Ar-B = 0. (5.57) 

This equation is called the characteristic equation. It determines two solutions 



A \A^ „ A W 



r,=-^-^B, r,=--^-^B. (5.58) 

The two roots I'l and rj are the eigenvalues of the characteristic equation (5.57). To 
prepare the following developments we write /-[ and r^ as 

where r^ and / -^ are given by 

J\+r^^A HZ^^I^ + B. (5.60) 

2 2 2 V 4 

This representation of r, and rj is helpful because of the explicit consideration of 
the square root of the term A^ / 4 + B (this term can be real, zero or imaginary). 
Hence, Eq. (5.53) for j„ satisfies the difference equation (5.45) provided I'l and r, 
are given by Eqs. (5.59). 

Consistency with Initial Data. Next, let us consider under which condition 
Eq. (5.53) for j,, is in consistency with the initial data y^ and j,. To address this 
question we set « = and « = 1 in Eq. (5.53), which leads to the conditions 

y^,=s + c^ + C2, (5.61a) 

J, = s + c^ r, + Cj r,. (5.61b) 

A relation for Ci can be found by multiplying Eq. (5.61a) with -rj and taking the 
sum of both equations. Similarly, a relation for C2 can be found by multiplying 
Eq. (5.61a) with -r, and taking the sum of both equations. In this way we find the 
relations 

yi-s-r2(yo-s) = Ci(r,-r2), (5.62a) 

yi-s- iCJo - •^) = -C2 ('"1 - '"2)- (5.62a) 
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The use of these relations for Cj and c, in Eq. (5.53) leads then to the solution 

„ _„, yj-s-r^iyp-s) ^^,, y^ - s - r,(ya - s) ^„ 
y„ - i + r^ Tj 



■ s + - 



-(yi-s)- 



'2'1 '1'2 



(yo-s) 



(5.63) 



'1 '2 
II n 



= s + 



W - ■^) - '1'2 - 



r, - }\ 



(j'o - ■^)- 



'1 '2 '1 '2 

This solution satisfies both the difference equation (5.45) and the initial conditions 
that j„ is equal to jo and j, at « = and « = 1, respectively. 



53.2 Solution Features 



Three Solution Cases. The solution y„ will have different features depending 
on the value of ^^ I A+B'\n I'l^. We have to distinguish the cases ^^ / 4 +5 > (r^ is 
real), A^ I A + B = Q {r^ = Q), and ^V 4 + _6 < (r^ is imaginary). The three cases 
will be discussed in the following. Regarding the first case {A^ I A+B > 0), there is 
no problem with the direct use of Eq. (5.63) for the calculation ofy„. A second 
case is given for ^^ / 4 +i? = 0. We have then r, = rj according to the definition 
(5.59) of r, and r,. It is impossible to use Eq. (5.63) directly for this case because 
of the need to divide zero by zero. The solution can be obtained from Eq. (5.63) 
by taking the limit r^ — > 0. The latter can be done by means of I'Hospital's Rule, 



y„=s + 






iyi-s)- 



'•d=o 



d{r" -r^ )ldrj^ 
d(ri -r^)l dr^j 



(jo - s) 



'»=0 



= s + n 



ii-\ ii-\ 



{y,-s)-{r,-\) 



'■Z)=0 



n-2 n-2 



(jo - s) 



'■Z)=0 



= ^ + « Y iyi-s)-{n-l){ — \ iya-s). 

(5.64) 
A third case is given \fA^ I A + B < 0, which means that r^ is an imaginary number. 
The eigenvalues r, and rj can be written for this case 

1 = rs + '■ '/)•: ''2^''s- ' '/)•■ (5-65) 

Here, / satisfies the equation f = -l, and we applied the abbreviation 



B. 



(5.66) 
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The rewriting of the solution (5.63) for this case can be handled by means of the 
de Moivre's formula (Abramowitz & Stegun 1984), 



(rj ± ;>£,.)" = K"{cos{np) + ;sin(«^)). 
Here, /rand (p are defined by the expressions 



^ = arctan(r^, /r;). 



-B, 



(5.67) 

(5.68a) 
(5.68b) 



The writing of «• ensures that «•= r^ for r^,, = 0, which follows from Eq. (5.67) for 
« = 1. The case considered implies B <- A^/4. Hence, k'is real-valued. The use of 
Eq. (5.67) for r/' and rj" provides 

r/' - Tj" = «'"(cos(«^) + ism{n(p))- K:"{cos{n(p) - /sin(«^)) = liic" sin(n^). 

(5.69) 
The use of the latter expression for n and «-l in Eq. (5.63) then implies 



y„=s. ." ^J^^klA^y^ _ ,) _ «- V"-' ^iM!LzM(,„ _ ,) 



S + K 



sin(«^) 



2ir„, 

(,,-.)-.-M^^^M(,„_,). 



(5.70) 



fi-,* 



In this relation we used r. 



2 / r^,, and r, r^ 



t^ according to 



Eq. (5.65). Consequently, for the three cases considered the solution >>„ is given by 



n-\ n-\ 



■-^{y^-s)-,Y,'-^ '^{y,-s) ifA'/4 + B>0 



y„-s + \nl^] (y^-s)-{n-l)(^] iy,-s) ifA'/4 + B = 0. 

^„sin(«^^^^_^^_^„,sin((«-M(3,^_,) ,fA'/4 + B<0 

(5.71) 
The cases^V4 + fi > 0, A^ / 4 + B ^ 0, and A'^ / 4 + B < will be referred to as 
case 1, case 2, and case 3, respectively. 

Illustration. Characteristic features of these different solutions are illustrated in 
Fig. 5.4. The initial values are given by yg ^ OA and y^ = 0.2. The parameters C 
and B are given by C = 0. 1 and B = -Bg A^ 1 4. The introduction of B^, allows us to 
differentiate between the three cases considered. A^ 1 4+B is then given by 



^-=('-''.)f 



(5.72) 
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Fig. 5.4. An illustration of solution features of the second-order linear difference 
equation (5.45) for the cases 1, 2, and 3 in (a), (b), and (c), respectively. 



Thus, the setting i?(, < 1,8^^= 1, and i?o > 1 corresponds to the consideration of the 
first, second, and third case, respectively. The parameter A is chosen such that the 
equilibrium solution .s = 2 is constant. This condition provides the equation 

C C C 



(5.73) 



\-A-B l-A + B„A'/4 l-l/B„+(A-2/B^YBJ4 

for the calculation of ^. By rewriting this condition as a quadratic equation for^, 
the parameter A is found to be given by 



A = - 



B. 



\B 



C 



■1 + 



V 



A 



(5.74) 



oy 



The solution features are illustrated in terms of Fig. 5.4 for three values ofB^ that 
correspond to the three cases considered. Figure 5.4b shows a curve that is similar 
to the curve in Fig. 5.1a (which shows a first-order equation solution). A differ- 
ence is given by the possibility to choose ji in the second-order equation solution. 
The latter fact enables the modification of the initial behavior of second-order 
equation solutions. Figure 5.4a shows a major difference between first-order and 
second-order equation solutions: the second-order equation offers the possibility 
to modify the transition to the equilibrium solution by the inclusion of rj" in 
addition to r" (see the first line of (5.63)), which is often helpful for the modeling 
of observations. Figure 5.4c demonstrates another major difference between first- 
order and second-order equation solutions: the second-order equation offers the 
chance to involve oscillations about the equilibrium solution (which may be seen 
as the result of two trends that act in opposite directions). The amplitude of such 
oscillations depends on the value of ic. see Eq. (5.70). For k< 1, k= 1, and k> \, 
the oscillations are damped out, have a constant amplitude, or an increasing 
amplitude, respectively. The value for k'iw Fig. 5.4c k k= 0.93. These features 
lead to the conclusion that the solutions of second-order equations can cover a 
much broader variety of cases than solutions of first-order equations. 
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Fig. 5.5. Model parameter regimes that are defined 
by Samuelson's national income model (5.75). The 
functions « = 4 /?/ ( 1 + /?)^ and « = 1 / /? are shown 
by the solid and dashed lines, respectively. 



5.3.3 National Income 

National Income Model. Let us come back to the national income model (5.5) 

y„=a{\ + p)y„_,-aPy„_,+\ (5.75) 

to illustrate the advantages of using second-order difference equations. The range 
of variation of the marginal propensity is < « < 1 . According to Samuelson, the 
range of variation of /? is < /? < 4, and we use y = 1 . By replacing all y„ in 
Eq. (5.75) by s and solving for s, we obtain for the equilibrium solution s 

1 



l-« 



(5.76) 



Hence, the equilibrium value is only controlled by a such that the transition to the 
equilibrium is controlled by /?. Such a separation of the role of model parameters 
as given here is very helpful for the adjustment of a model to any given data. 

Solution. The simplest way to find the solution for the national income model 
is to relate a and y^in (5.75) to A, B, and C in Eq. (5.45). The comparison of these 
equations shows that 



A^a{\ + p), 



B = -aP, 



C = l. 



(5.77) 



We calculate A^ I A+BXo see which of the cases considered above is given here, 



a 



+ B = =—{\ + py -aP 



^(1 + /?)^ 



a 



Ap 



{l + Pf 



(5.78) 



This relation shows that all three cases considered above can be realized: A^ I A + B 
can be positive, negative, or equal to zero depending on the relation between 
a and /?. The behavior of oscillations is determined by the value of k. The use of 
the K definition i^ = -B combined with B = -a P shows that 



ap. 



(5.79) 
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Fig. 5.6. Characteristic solution features of Samuelson's national income model 
(5.75). The model parameters apphed are (a) «= 0.5, /?= 0; (b) « = 0.5, /?= 2; (c) 
« = 0.6, y^ = 2; and (d) a = 0.8, P = 4. 



Thus, the oscillations will increase or decrease depending on whether « /? > 1 or 
a P < \, respectively. The various model parameter regimes of Samuelson's 
national income model (5.75) are illustrated in Fig. 5.5. The areas A and D have 
case-1 type solutions. The areas B and C have type-3 case solutions. The state 2 is 
shown by the solid line a^ A fil (\+ Pf. The curve a^\l fi corresponds to «• = 1 
that separates the regime B (where oscillations are damped out) from regime C 
(where the amplitude of oscillations increases). 

Solution Features. The features of solutions of Samuelson's national income 
model are illustrated in Fig. 5.6, where jo " and ji = 1 are applied in all cases. 
Samuelson identified four regimes of solution behaviors that are characterized by 
the corresponding a and fi values applied in Fig. 5.6. Figure 5.6a illustrates the 
case that the national income approaches asymptotically a constant level 1 /(I - a). 
Figure 5.6b illustrates oscillatory national income movements with a constant am- 
plitude {k=\ for this case). Figure 5.6c illustrates increasing oscillatory motions 
{k> 1), and Fig. 5.6d illustrates a national income with an increasing growth rate. 
Discussions of the economic relevance of these model features can be found 
elsewhere (Samuelson 1939). 
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5.3.4 Red Blood Cell Production 

First-Order Equation System. Let us consider a model for the number of red 
blood cells (which carry oxygen throughout the body) circulating in the blood. 
The model considered is the following one (Edelstein-Keshet 2005), 

^„=(l-/R->+K-i, (5.80a) 

M„=c^fR„_,. (5.80b) 

R„ refers to the number of red blood cells on day n. Two contributions can change 
R„. The first contribution (1 -f)R„-i is proportional to the current number 7?„_, of 
red blood cells. The factor 1 -/accounts for the fact that the spleen filters out and 
destroys a certain fraction of the red blood cells daily. Here,/> is the fraction of 
red blood cells removed by the spleen. The second contribution A/„_, accounts for 
the production of red blood cells by the bone marrow. Equation (5.80b) specifies 
the production Af„. This equation states that the production is proportional to the 
number of red blood cells f R„-\ lost on the previous day. A parameter c^ is 
involved in Eq. (5.80b) to account for variations of the production M„. 

Second-Order Equation. It is convenient to reformulate the model (5.80) to 
use the solutions of equations derived above. By taking Eq. (5.80b) at « - 1 we 
find M„_, = c^fR„_2. Hence, Eq. (5.80a) can be written 

^„-(1-/R-i+Cm/^„_2. (5.81) 

This equation is a second-order linear difference equation. It has the structure of 
the second-order model (5.45): for the case considered we have j„ = R„, A^ \ -f, 
B = c^f, and C = 0. Hence, in difference to the national income example we have 
now a case where B > 0. 

Model Solution. The solution to the model (5.81) can be found by means of the 
solution (5.71) of the difference equation (5.45), where A^ \ - f, B = c^f, and 
C = 0. According to Eqs. (5.58), the eigenvalues ri and r, are found to be given by 



1 

r, = — 
' 2 



1-/ + V(1-/)'+4cm/J, r,=-[l-/-7(l-/)^+4c,,/ 



(5.82) 



For c^f > as considered below, the term (1 - fY + 4 c^f in the square root is 
found to be positive. Hence, we have two different and real eigenvalues }\ and rj. 
Thus, the solution to Eq. (5.81) is given by Eq. (5.63), 

K = '^^^^R, - r,r, ''"'"'^"' ^0. (5.83) 

The equilibrium solution s is zero here because of C = for the case considered. 
The parameters ^o and T?, refer to R„ at « = and n= \, respectively. 
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Fig. 5.7. The variation of the eigenvalues /-[ and r-^ with the model parameter Cf^ 
according to (5.82), where/= 0.01 is applied. 



Production Model Specification. The model behavior depends essentially on 
the eigenvalues r\ and r^- An example for the variation of the eigenvalues r\ and r^ 
with c^^ is shown in Fig. 5.7 for/= 0.01. It may be seen that r^ is always negative 
and small. Hence, the effect of Tj on the solution features will be rather limited: r{' 
will disappear for growing n. The r^ plot shows that }\ may be smaller (for c^ < 1), 
equal (for c^ = 1), or larger than one (for c^ > !)• For r, values smaller (or larger) 
than one, r" will constantly decrease (or increase) with a growing n, this means 
the number R„ of red blood cells will constantly decrease (or increase). A model 
that does always predict a decreasing or increasing number of red blood cells does 
not make a lot of sense. Thus, we will only consider the case r^ = \ (which is 
given for c^^ = 1). Such a model may lead to a relatively constant number of red 
blood cells. We consider, therefore, in the following the model 



^„-(l-/R-l+/^„-2- 



(5.84) 



Model Analysis. For this model, Eqs. (5.82) for the eigenvalues r, and r^ can 
be simplified in the following way. 



i-/+V(i-/)'+4/ =- i-/+V(i+/)' 



= 1, 



1-/-a/(1-/)'+4/J--^1-/-V(1 + /)' 



-/■ 



(5.85a) 
(5.85b) 



According to Eq. (5.83), the solution for7?„ is given for this case by 



R 



!-(-/)" ;. , ^1- (-/)'"' ;, _ ^> , /^o , (-/)" 



1 + / 

R. 



-R,+f- 



1 + / 



-R. 



\ + f \ + f \ + f 



1 + / 



(i+/-i)^o.,(zzr(^^ 



1+/ 



1+/ 



-R,] = R. 



Ri -Rq 
1 + / 



(^0-^,) 



C_ f\" ' 



1 + / 
(5.86) 
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Fig. 5.8. Solutions R„ of the red blood cell production model (5.84), where the 
initial values R(,= I and 7?, = 0.9 are used. In (a), (b), and (c), the values /= 0.01, 
/= 0. 1, and f = 0.5 are used, respectively. 



For large n, (-/)" will disappear. Hence, the equilibrium solution R^ reads 



^ = ^n + - 



1 + / 



(5.87) 



It is worth noting that this expression cannot be found immediately by replacing 
R„ by R^ in Eq. (5.84): we end up with the identity R^ = R^ in this way. Regarding 
the following discussion it is helpful to rewrite the solution (5.86) of the red blood 
cell model (5.84) by taking reference to the equilibrium solution (5.87), 



R„=K-i-fy{K-Ro)- 



(5.88) 



Due to the appearance of the factor (-J)", the solution 7?„ does oscillate about the 
equilibrium solution R^ until R^ is reached. The reason for these oscillations is the 
time delay considered: the bone marrow produces red blood cells proportional to 
the number of cells lost on the previous day. 

Model Applicability. The suitability of the model (5.84) is considered in terms 
of Fig. 5.8. For a reasonable value/= 0.01, this figure shows that the transition to 
the equilibrium value takes place very fast (see Fig. 5.8a). At n = 1, there is only a 
very little difference to the equilibrium solution (7?, = 0.9 and R^ = 0.901). At 
n = 2, we have, basically, the equilibrium value (7?, ^ R^ + 9.9 x 10 ''). Such a 
model behavior is clearly unreasonable. It requires unreasonable values of/= 0.1 
and/= 0.5 to slow down the fast transition to the equilibrium. Another interesting 
finding is that the model (5.84) cannot respond to perturbations. Let us consider 
the value ^i = 0.9 as a perturbation (which may be the result of a donation of 
blood). Then, there is no homeostasis mechanism such that R„ can return to the 
previous value of one. Due to these reasons, the model (5.84) represents a poor 
model for the production of red blood cells. It is worth noting that is requires 
much more complex models to deal with reality (Yafia 2009). 
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5.4 Nonlinear Changes 

The third equation type that we will discuss is a nonlinear first-order difference 
equation. The initial value y^ at « = is considered to be given. The y„ values for 
« = 1,2, ... are determined by the equation 



y„ =/(j„-i)- 



(5.89) 



Here, the right-hand side/(y„_i) represents any given (linear or nonlinear) function 
of j„_|. The addition of - y„_i on both sides shows that this equation determines 
changes j„ - j„_i. Equation (5.89) is nonhomogeneous if/(y„_i) adds any constant 
(as given, for example, for the case /(y„_i) = >'„_i^ + 1). We have a first-order 
difference equation because only function values at n and « - 1 are involved. 



5.4.1 Analysis Concepts 



A significant difference between linear and nonlinear difference equations is 
that analytical solutions of nonlinear equations are usually unavailable. Hence, we 
have to use other methods to analyze such equations. 

Pendulum. Let us consider an example in order to illustrate the basic concepts. 
Figure 5.9 illustrates the motion of a pendulum, where a denotes the angle that the 
pendulum makes with the vertical direction. A detailed mathematical explanation 
of the pendulum motion is not really simple: it requires the use of a nonlinear dif- 
ferential equation (see the discussion of this problem in Sect. 9.4). Nevertheless, 
we can determine the characteristic features of 
pendulum motions without the application of 
relatively advanced mathematical tools. First 
of all, it is helpful to see which final state the 
system will realize. The latter question can be 
addressed in two steps. The first step is to find 
equilibrium solutions (this means stationary or 
steady state solutions). Figure 5.9 shows that 
there are two equilibrium positions of the pen- 
dulum motion: the position 1 with the mass 
directly below the point of support (i.e., for 
a = 0), and the position 2 with the mass directly 
above the point of support (i.e., for «= ;r). The 
second step is to clarify whether the system 
will realize the potential equilibrium solutions. 



Position 2 (unstable equilibrium) 

, O 



i 



■ o •' 

Position 1 (stable equilibrium) 
Fig. 5.9. A pendulum. 
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The best way to find the answer to this question is to consider motions relatively 
close to the equilibrium points and to ask whether the equilibrium points will be 
realized or not. An equilibrium point is called stable if the system will realize this 
equilibrium point, otherwise the equilibrium point is called unstable. With regard 
to the equilibrium points shown in Fig. 5.9 we see a different behavior. If the mass 
is slightly displaced from the lower equilibrium position 1, the pendulum will 
oscillate back and forth with gradually decreasing amplitude until the equilibrium 
point is finally realized. Such an equilibrium point is stable (it attracts the motion). 
If the mass is slightly displaced from the upper equilibrium point 2, it will rapidly 
fall and the mass will finally approach the equilibrium point 1 . Thus, the second 
equilibrium point is unstable. 

Equilibrium. How is it possible to find equilibrium solutions of any system? 
A steady state means that y„ does not depend on n anymore, this means we have 
yn "= yn-i " ye, where y^ refers to the equilibrium solution. According to y„ =/(y„_i), 
the condition for finding y^ is given by the equation 

y.-f(y.)- (5.90) 

This equation may well have several solutions if we have to deal with a nonlinear 
equation. 

Stability. How is it possible to decide whether any equilibrium point is stable 
or not? To address this question we have to consider y„ relatively close to any 
equilibrium point, 

J„-J.+"„- (5.91) 

Here, u„ represents a little deviation from the equilibrium solution. By making use 
of this expression my„ =/(y„_i) we find 

J. +«„-/(j. +«„-.)■ (5.92) 

Now, we account for the fact that u„ is small. In particular, we expand the right- 
hand side in terms of the Taylor series about y„ where terms that are nonlinear in 
M„ are neglected, 

ye + i'..-f(y.) + f(y>„-v (5.93) 

Due to the definitions'^ ^fiye) of the equilibrium solution, this equation reduces to 
".,=f(y>„-,- (5.94) 

This equation is a homogeneous linear first-order difference equation. According 
to the discussion in Sect. 5.2, u„ will disappear for large n if 

|/'(jj|<l. (5.95) 

This inequality represents the condition for a stable equilibrium point: the solution 
will approach the equilibrium point if this inequality is satisfied. 
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5.4.2 The Discrete Logistic Equation: Analysis 

Discrete Logistic Equation. As an example, let us consider a quadratic first- 
order difference equation 

y„=Ay^_,+ByJ+C, (5.96) 

where A, B, and C are any model parameters. In particular, we will assume that 
A = 1 +a,B = -alK,?inAC = 0, 

« „ 2 _ ,, , „,, r, yn-i 



y„={l + a)y„_, --;;„_;= ^„_, + a j„_,^l-^ J. (5.97) 

Equation (5.97) is called the discrete logistic equation or the logistic difference 
equation. The assumptions for A and B represent appropriate rewritings of ^ and B 
in terms of two other parameters a and K, and the assumption C = simplifies the 
following analysis. A closer look at Eq. (5.97) reveals the suitability of the 
assumptions A = \ + a and B = - a I K. We see that one equilibrium solution is 
given for the case that _v„_i = K\ the parenthesis term is zero for this case, which 
means that >>„ = y„_^ for all n. A second equilibrium solution is given for j„_i = 0, 
which does also result in j„ =>'„_i for all n. We are interested to ensure thatj,, > 
for all n such that the model considered can be used, for example, as a population 
model. Thus, we assume that y^ > 0, and we also assume that a > and K>0. The 
consideration of negative K values would mean that y„ can realize a negative 
equilibrium solution. The consideration of negative a values would mean that y„ 
will approach zero or infinity for large n depending on whether the parenthesis 
term in Eq. (5.97) is positive or negative initially, respectively (a negative a value 
can also imply oscillations of j„ about zero including the appearance of negative j„ 
values). Thus, the equilibrium value K cannot be realized if a < 0. 

Nondimensional Discrete Logistic Equation. Instead of working directly with 
Eq. (5.97), it is helpful to analyze a nondimensional version of this model. For 
doing this, we write the middle expression in Eq. (5.97) as 



J,, =(1 + «)>-„- 



1 — y 



(5.98) 



Now, we introduce the variable 

The variable x„ is nondimensional because it is compared to one in the bracket 
term of Eq. (5.98). By multiplying Eq. (5.98) with a/ [(1 +a) K\ we obtain 

^„ =/(^„-i ) = (! + «) ^Vi(l-^„-i) ■ (5.100) 

This equation is called the nondimensional discrete logistic equation. 
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Parameter Variations. We are interested to ensure that x„ > for all n, which 
implies that y„ > for all n. The condition x„ > requires that x„_i < 1 : otherwise x^ 
will become negative according to Eq. (5.100). To see the upper bound of x„_,, we 
need to know the maximum of x„ for all n. This maximum can be obtained by 
calculating the maximum of/(x„_|). The derivatives of/(x„_,) are given by 

^^ -(l + fl)(l-2x„_i), -^-^ = -2(l + fl). (5.101) 



Hence, /(x„_i) has a maximum value at x„_i = 1/2. The maximum of/(x„_i) at 
x„_, = 1/2 is given by 

/(l/2) = maxx„ =(l + a)/4. (5.102) 

Consequently, the condition a < 3 ensures that max x„ < 1 for all n. As a conse- 
quence of this analysis, the variation of a to be considered is given by 

0<a<3. (5.103) 

This condition ensures < x„ < 1 for all n. 

Equilibrium. To find the equilibrium solutions of the nondimensional discrete 
logistic equation, we replace x„ and x„_i by x^ in (5.100), 

x^ = (1 + a)x^ (1 - x^) - x^ + ax^ - (1 + a)x^ . (5.104) 

The corresponding quadratic equation for x^ can be written 

= x^(fl-(l + a)xj. (5.105) 

Hence, we find two equilibrium solutions x/" and x^*^', 

x;'>=o, x;^'-T^. (5.106) 

l + a 

Stability. The stability of these equilibrium points can be evaluated according 
to the inequality (5.95). According to Eq. (5.101), the derivative/' at the equilib- 
rium points x^''' and x^'^' is given by 



/'(x'") = l + a, /'(x'") = (l + «) 1-2 =l + fl-2a = l-fl. (5.107) 

V 1 + flj 

The stability condition [f' (x^)] < 1 then implies for the equilibrium solutions x/'' 
and Xj,'"' the stability conditions 

x/': -2<a<0, x/" : < a < 2. (5.108) 

The first equilibrium solution x/'' is an unstable equilibrium point for the a values 
< a < 3 considered. Regarding x^*^' we find the following behavior, 

0<a<2: stable 2<a<3: unstable (5.109) 
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Fig. 5.10. An illustration of solutions of the nondimensional discrete logistic 
equation (5.100) in dependence on n, where y^ = 0.01. The corresponding values 
of the model parameter a are shown in the figures. 

At a = 2 the stability properties of the equilibrium solution x/^' change. The values 
of fl for which such changes occur are called bifurcation values. 



5.43 The Discrete Logistic Equation: Solutions 



The features of numerical solutions of the nondimensional discrete logistic 
equation (5.100) are illustrated in Fig. 5.10, where >'o = 0.01 is applied. There are 
several solution regimes. 

Stable Growth. One solution regime is given for < a < 2. For relatively low a 
values, < a < 1, we observe a smooth solution. Initially, there is an exponential 
increase, but then the curve levels off The asymptotic value is determined by 
x/^' = a l{\ + a). For larger values I < a < 2 the increase is so rapid that the 
solution overshoots the equilibrium value. A period of damped oscillations fol- 
lows before the solution converges to the equilibrium value x^'^' = a /(I + a). The 
solution behavior for the value a = 2 is shown in Fig. 5.10c. This figure shows 
again damped oscillations, but the equilibrium point x/^' = 2/3 cannot be realized 
for finite n (which means that the solution shows an unstable behavior). 
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Cyclic Growth. Another solution behavior is observed for 2 < a < 2.57. The 
solution shows oscillations which are no longer damped: after an initial stage the 
solution shows bounded oscillations. In Fig. 5.10d, the bounds are given by 
0.4794 and 0.8236. The solution has a cycle: after the initial stage the solution 
repeats itself after two intervals (i.e., the solution is the same between « = 30 and 
n = 12, and between n =32 and n =34). This cycle is called a 2-cycle. For a =2.5, 
the cycle changes to a 4-cycle (for every four values the solution is the same), and 
there are higher-order cycles for higher values of a. 

Chaotic Growth. However, there is a limit for such cyclic growth at a « 2.57: 
for higher values of a there is no cyclic growth, but the growth patterns become 
random, i.e., we find a chaotic solution behavior for 2.57 < a < 3. Figure 5.10f 
illustrates the problem with values of a bigger than 3 : the solution may become 
negative in these cases. The first systematic studies about the chaotic behavior of 
solutions of the logistic difference equation where made by May (1975, 1976). 
The number a « 2.57 was discovered by Feigenbaum (1978). Thus, this value is 
called the Feigenbaum number. A nice discussion of investigations of properties 
of the logistic difference equation can be found in the book of Gleick (1987). The 
textbooks of Edelstein-Keshet (2005) and Allen (2007) also provide very helpful 
insight. 



5.5 Difference and Differential Equations 

The purpose of this section is to illustrate some advantages and disadvantages 
of difference equations. It will be shown that the use of difference equations faces 
several problems related, for example, to the parameters of difference equations, 
the interpolation between data values, and the analysis of equations and solutions. 
Such problems can be overcome by the application of differential equations. The 
latter does not mean, however, that difference equations are not useful. Instead, it 
will be shown that difference equations play a very important role: they are used 
for the formulation of differential equations, the numerical solution of differential 
equations, and in some cases even the analytical solution of differential equations. 
The common use of differential and difference equations is such that laws (e.g., 
the laws of mechanics, population ecology, and probability theory: see the discus- 
sion of these equations in Chaps. 7-10) are usually formulated in terms of differ- 
ential equations. Such laws are often represented in terms of nonlinear equations 
that cannot be solved analytically. The application of difference equations is then 
the only way to find numerical solutions to such nonlinear differential equations. 
Therefore, difference equations definitely represent a very valuable concept. 
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Fig. 5.11. A comparison between difference and differential equations regarding 
the U.S. population development between 1790 and 1890. The population data are 
shown as solid symbols. The open symbols show the solution of a difference 
equation, which depends on the distance A? of data points. The solid line shows 
the solution of a differential equation, which does only depend on the process 
parameter r and initial data. 



5.5.1 Difference Versus Differential Equations 

U.S. Population Data. Let us consider again the U.S. population data from 
1790 to 1890 discussed in Chap. 1 to illustrate a few questions regarding the use 
of difference equation. The observed population data are presented in Fig. 5.11. 
According to expression (1.19), the U.S. population data can be well described by 
the function 



■-P,e 



(5.110) 



The parameters of this exponential model are given by Pq " 0.0039, ?o " 1790, and 
the characteristic time scale is r= 35 years. 

Difference Equation Model. First, let us design a difference equation model in 
agreement with the U.S. population data. A simple way to derive this model is to 
rewrite the known model (5.110). We introduce t - 1^ = n IS.t. Here, A? is the time 
step considered, which is 10 years for the U.S. population data, and « = 0, 1, .... 
The exponential population model can be written then 



P„=P,a", 



(5.111) 



where a = exp(A?/ z). This expression represents the solution of a first-order linear 
difference equation that is given by (see Sect. 5.2) 



P„ = a P., 



(5.112) 
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Differential Equation Model. A differential equation model can be found cor- 
respondingly. The differentiation of (5. 1 10) leads to 

— = ^e^. (5.113) 

dt T 

The use of P given by (5. 1 10) on the right-hand side enables the rewriting 

^ = ^. (5.114) 

dt T 

This equation represents a differential equation model for P. The solution for this 
model is given by the exponential model (5.110). 

Problems of Difference Equations. The comparison of the solution (5.111) of 
the difference model (5.112) with the U.S. population data in Fig. 5.11a reveals 
several problems. First, in reality we often have continuous processes, but obser- 
vations about these processes are only given for a certain resolution (i.e., for a 
certain A? that is ten years for the population data considered). Thus, usually we 
are interested to have information about the process between the observed data 
points. A difference equation that is designed such that it agrees with observed 
data does not provide such insight. Second, it is often helpfiil to be able to use 
models analytically to calculate, e.g., the minimum or maximum of fiinctions. The 
structure of the solution of difference models makes it usually difficult to use such 
solutions analytically. Third, actually we look for models that are as universal as 
possible, this means we are interested in models that can be applied with the same 
model parameters to many different cases. Difference equations do not represent 
such universal equations: we have to use different equations for every distance 
between data values. The latter can be seen by considering the expressions (5. Ill) 
and (5.112), which depend on A?. By adjusting the difference equation to the data 
set as considered in Chap. 1 (data values at 1790, 1800, 1810, ...), we have a = 
exp(10/35) « 1.33 for A? = 10 years and x = 35 years. Alternatively, we could use 
the exponential model (5. 110) such that the distance of function values is only one 
year (values at 1790, 1791, 1792, ...). For this case, we have A? = 1 such that the 
parameter a would be a = exp(l/35) = 1.029. In general, we could try to fix this 
problem by accounting for the resolution effect (the setting of At) on the model 
parameters. Such an approach may work for a simple case as considered here, but 
it is hard to use for more complex cases. 

Difference Versus Differential Equations. These problems can be overcome 
by the use of differential equations: see the illustration in Fig. 5.11b. Differential 
equations provide continuous solutions for all t, it is often possible to use such 
solutions for analytical investigations, and the parameters of differential equations 
are independent of the resolution of processes. Nevertheless, this does not mean 
that difference equations do not represent a valuable concept. Difference equations 
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are used for the formulation of differential equations, their numerical solution, and 
even their analytical solution. These advantages of difference equations will be 
illustrated in the next two subsections regarding the solution of first- and second- 
order linear differential equations. 



5.5.2 First-Order Difference and Differential Equations 

Problem. Let us consider first a linear first-order differential equation 

± = -1^1^. (5.115) 

dt T 

The constant y^ represents the equilibrium value of j (for y^y^ there is no change 
of J' anymore). The constant T refers to a characteristic time scale that determines 
the transition between any initial value ^(O) and y^. A discussion of the physical 
relevance of this equation can be found in Sect. 7.2. Equation (5.115) is defined as 
limit A? — > of the difference equation 

Yn - Yn-l ^ _ J„-l - y^ ^ (5.116) 

At T ' 

This equation represents the first-order linear difference equation (5.6), where 
a = - Kt I T and h = y^ ht I T. In the following, we will determine the analytical 
solution of Eq. (5.1 15) on the basis of the difference equation (5.116), and we will 
show that the numerical solution of the differential equation (5.115) can be found 
by means of the difference equation (5. 1 16) if a sufficiently small A? is used. 

Analytical Solution. The solution (5.15) of the difference equation (5.6) shows 
that the solution of the Eq. (5. 1 16) is given by 

The last expression makes use of a = 1 - A^/Tand h ^ y^. At/ T, and>'o refers to the 
initial value of y„ (as may be seen by setting n = 0). To find the limit A? — > of 
this expression we consider first 

Here, t ^ n At is used, and the logarithmic function is approximated by its Taylor 
series in the first order of approximation, 

lnfl-f] = -f. (5.1.9) 
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Fig. 5.12. The convergence of numerical solutions of the differential equation 
(5.121); (a) the analytical solution; (b) the normalized error e I ht of numerical 
solutions of Eq. (5.121) in dependence on the time step A?. 



Hence, the solution (5. 117) reads in the limit A? — > 

y{t) = y^+e'"\y^-y;). 



(5.120) 



where j„ is replaced by the continuous function j(0. 

Numerical Solution. It is often the case that analytical solutions of differential 
equations cannot be found. In such a case, we have to find numerical solutions of 
differential equations via the solution of a difference equation. To illustrate the 
correctness of such numerical solutions for a sufficiently small A?, we consider the 
equation 

dy _ y-\ 
dt 2 



(5.121) 



combined with the initial condition Jq ^ 0.1. The analytical solution (5.120) of this 
equation is shown in Fig. 5.12a. The numerical solution of Eq. (5.121), which is 
obtained via the difference equation (5.116), depends on the time step A?. To 
assess the error of the numerical solution we introduce the standardized error 



yn-yit) 



yit) 



(5.122) 



This error considers the absolute value of the normalized difference between the 
solution jv„ of the difference equation (5.116) and the analytical solution XO given 
by (5.120). The maximum is taken over of all absolute values with < ? < 20. Fig. 
5.12b shows that the normalized error e/ A? is found to be independent of A? for 
increasing At. In particular, we find 

e = 0.1521A? (5.123) 

for M < 10 ^ Hence, the solution of the difference equation (5.116) represents an 
accurate tool to solve the differential equation (5.115) numerically. 
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5.5.3 Second-Order Difference and Differential Equations 
Problem. Next, let us consider the linear second-order differential equation 

a^ + b^ + cy = 0, (5.124) 

where a, b, and c are any model constants. This equation will be used in Chap. 7 
for the explanation of laws in mechanics. We define this equation as the limit 
A? — > of the difference equation 

= ^ ^""-"-' - ^--"^"-4 + fe ^-'"^"-^ +c^„,. (5.125) 

At[ At At J At '"' 

The first term represents the second-order derivative defined as derivative of first- 
order derivatives. The second term can be justified by setting c = 0. For this case, 
Eq. (5.125) provides a correct first-order equation forx„ = (y„ - j„_,)/A?. The third 
term can be justified by setting a = 0, which results in a correct first-order equa- 
tion. To compare Eq. (5.125) with the second-order difference equation (5.45) we 
write Eq. (5.125) as 

b c 

= j„ - 2j'„-, + y„-2 +-^t(y„-i - y„-2) + - a?" j„_2, (5- 126) 

a a 

where a ^^ is assumed. The latter equation can be written 

y„=Ay,,_,+By,,_„ (5.127) 

where A and B are given by 

^ = 2--A^ B^-l + -At--At\ (5.128) 

a a a 

Reformulated Problem. The second-order difference equation (5.127) can be 
also formulated in terms of a system of two first-order difference equations, which 
will be relevant regarding the discussion below. To obtain this system of two first- 
order difference equations we consider the variable x„ that is defined by 

^""/"'' =x„-i- (5.129) 

At 

By using this definition of x„ we can write Eq. (5.125) as 

Q^Q ^"-'~^"-2 +bx„ .+cy„2- (5.130) 

At 

We can replace « - 1 by « in this equation. The combination of the corresponding 
equation with Eq. (5.129) results in a system of first-order difference equations. 
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which is given by 

At 



At 



-y„-i 



(5.131a) 
(5.131b) 



Analytical Solution. To find the analytical solution of the differential equation 
(5.124) on the basis of the solution of the difference equation (5.127) we have to 
calculate the eigenvalues r^ and r, related to Eq. (5.127). Therefore we consider 



4 



B 



4-4 — At + ^-At- 
a a 



-l + -At- 
a 



-At' 



At^ 
4a' 



{h^-4ac). (5.132) 



According to Eq. (5.58), the eigenvalues r, and Tj are then given by 
b 



, „ ^ A? b' 

r. =1 At + — 

2a 2 



-4ac 



■.l-±At± — ^b'-4ac, 
2a 2a 



■l- 



2a 



-At- 



At \b--4ac 

~2 



(5.133) 



b_ 

2a 



■-\-^^At + :^W 



At 
~2a 



4ac. 



Here, the positive sign in the ri expression (the negative sign in the rj expression) 
applies to the case that the parameter a is positive. Consequently, we obtain two 
eigenvalues: one eigenvalue (which we will call r^) involves the square root with a 
positive sign, and another eigenvalue (which we will call Tj) involves the square 
root with a negative sign. Hence, the eigenvalues read 



ri-l + Sj At, 



where .^i and .^2 are defined by 
1 



2a 



-b + 4b^ 



■4ac 



r^ -\ + $2 At, 



J_ 
2a 



(5.134) 



■Vfo^ 



-4ac 



(5.135) 



According to Eq. (5.63) the solution of the equation (5. 127) is then given by 



y.-'-'^^r, 



yizHI^,." 

'2 ■ 



(5.136) 



By using the relations Ti = 1 + Si At and rj = l+S2At we can write the coefficients 
of r/' and rj" in this solution as 

y^-(l + S2At)yo _ yi-yo-S2Aty„ _ y\At-S2Aty^ _y\-S2yo 



{s^-S2)At 



{s^-S2)At 



{8^-82) At 



j^, -(l + .SiAOj'o _y^-ya-s^Aty^ _y\At-s^Aty^ .j'o-'^iJ'o 



(5.137) 



{s^-S2)At 



{3^-82) At 



isi-S2)At 
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Fig. 5.13. The convergence of numerical solutions of the differential equation 
(5.142); (a) the analytical solution; (b) the dashed line shows the normalized error 
e / A^ of numerical solutions of the discrete second-order equation (5.127); the 
solid line in (b) shows the error of numerical solutions of the system (5.131) of 
two discrete first-order equations. 



where the abbreviation y\ = {y\ - jo) '' ^t is introduced for the initial derivative. 
Hence, the solution of the second-order difference equation (5.127) reads 



J/i ~ 'l '2 



(5.138) 



In the limit A? — > 0, we can use the first-order approximation of the Taylor series 
of exponential functions to replace r, = \+SiM and rj^ l+Si^t by 

r, =e"^', r2=e'^^'. (5.139) 

Therefore, we find the solution (5.138) in the limit A? — > to be given by 



_ y ■^2 Jo s^l _ Jo ■^1 Jo S2t 



(5.140) 



Here, t = n M is used, and;',, is replaced by the continuous function j(?). This solu- 
tion is real 'ifb^-Aac>Q such that ^i and S2 are real. In correspondence to the 
solutions (5.71) of the second-order difference equation (5.45) it is possible to 
obtain also real solutions for Z^^ - 4 a c < by a rewriting of Eq. (5.140): see 
Sect. 5.3.2. Regarding the following discussion it is interesting to note that the 
solution (5.140) does only allow zero or infinite asymptotic solutions depending 
on whether S2 and ^'2 are negative or positive. Equations (5. 135) show that the case 
i'l = and S2 < 0, which would provide a solution that levels off at a nonzero 
value, is only given for a = or c = 0, i.e., we would not consider a second-order 
differential equation in this case. 
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Numerical Solution. As done with regard to the linear first-order differential 
equation considered before, let us illustrate how a linear second-order differential 
equation can be solved numerically by means of difference equations. To have a 
fair comparison to the result obtained above for the numerical solution of a first- 
order differential equation we are interested in a solution that looks similar to the 
solution of the first-order differential equation (a solution that levels off at one). 
As concluded above, the second-order equation (5.124) does only have zero or 
infinite asymptotic solutions. Thus, we will consider the function j = x + 1, where 
X satisfies the second-order differential equation 

d X , dx 
— ^ + 4— 
dt dt 

Hence, the equation for j reads 

d Y ,dy 
— f + 4^^ 
dt dt 

This equation has the desired property that y levels off at one. It will be combined 
with the initial conditions XO) = 0.1 and dy/ dt(0) = 1. Figure 5.13a shows that the 
analytical solution of Eq. (5. 142), which is given by j = x + 1 where x is found in 
terms of Eq. (5.140), looks similar to the solution of the first-order differential 
equation shown in Fig. 5.12a. Figure 5.13b presents the normalized error e / At of 
numerical solutions of Eq. (5.142), where e is defined by Eq. (5.122). It turns out 
that the consideration of the second-order difference equation has a significant 
influence on the convergence behavior of the solution. The application of the dis- 
crete second-order difference equation (5.127) leads to a numerical instability for 
At< 10 ''. On the other hand, the system (5.131) of two discrete first-order equa- 
tions provides a convergent solution 



2 +4— + 2x = 0. (5.141) 



2 +4^ + 2(j-l) = 0. (5.142) 



e = 0.4610 A? (5.143) 

for At < 10"*. Actually, the choice At < 10 ' is also sufficient to integrate the 
second-order differential equation accurately: e I At = 0.4617 at At = 10"^ is very 
close to the asymptotic value el At = 0.4610. Thus, the difference equation system 
(5.131) represents an accurate tool to solve Eq. (5.124) numerically. 



5.6 Summary 

Difference equations are used for two purposes: the modeling of observations 
given at points with a finite distance (e.g., population data that are available every 
10 years) and for the solution of differential equations. Let us summarize the basic 
observations made in this chapter with regard to these two applications. 
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Application to Modeling. The evaluation of models that are obtained by the 
use of difference equations corresponds to the process apphed to the evaluation of 
other models: see Sect. 1.4.3. On the other hand, the development of difference 
equation models faces some specific questions related to the type of difference 
equations considered, the modeling scenarios that can be covered in dependence 
on the range of model parameters considered, and the adjustment of model param- 
eters to observations. Let us have a look at these closely related questions. 

• First, which difference equation should be used for the modeling of any case? 
It is often reasonable to consider first a linear first-order equation provided there 
is basically one trend that has to be modeled (see, e.g., the trend shown in 
Fig. 5.1a). Usually, there are model deficiencies because a linear first-order 
equation does not offer a lot of flexibility. Such shortcomings can be addressed 
by using the more flexible concept of a second-order equation to modify the 
transition to an equilibrium state provided by a linear first-order equation (see, 
e.g.. Fig. 5.4a). The use of nonlinear models is an efficient tool to describe 
competitive trends that affect changes of variables (see, e.g., the solution of the 
logistic population model shown in Fig. 5.10a: this curve reflects the competi- 
tion of an exponential increase and an asymptotic trend to leveling off). 

• Second, which range of variations of model parameters should be considered? 
This question is as relevant as the choice of the difference equation: we have to 
know which range of model parameters should be considered to simulate a 
desired or observed scenario. The answer to the latter question depends on the 
equation considered. There is not a big problem regarding the use of a linear 
first-order equation: the dependence of model features on the model parameters 
is obvious: see Fig. 5.1. Regarding the use of a linear second-order equation it 
needs an analysis of the dependence of eigenvalues on the model parameters 
(see the discussion of solution regimes of Samuelson's national income model). 
The understanding of basic solution features of a nonlinear equation requires an 
analysis of available equilibrium points and their stability. 

• Third, how can we adjust model parameters in difference equations to obtain an 
optimal agreement with observations? There is usually no unique way to derive 
all the model parameters from any data set. A helpful way is the use of relations 
that ensure that the modeled equilibrium solution agrees with observations. For 
example, the setting of the equilibrium value ^• = ^7/(1 -a) in a first-order equa- 
tion or the setting of s = C / (I - A - B) in a second-order equation provides 
conditions for the model parameters. Other model parameters have to be deter- 
mined such that the modeled transition to an asymptotic state optimally agrees 
with observed trends. An appropriate definition of model parameters supports 
the analysis of effects of parameter variations (as given, e.g., by the use of a and 
yffin Samuelson's model where «does fully control the equilibrium solution). 
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Application to tiie Solution of Differential Equations. The most relevant use 
of difference equations is their use for the formulation of differential equations, 
the numerical solution of differential equations, and the analytical solution of dif- 
ferential equations. Two examples for the application of difference equations to 
the solution of differential equations were presented in Sect. 5.5. Several other 
examples for the usefulness of difference equations will be given in Chap. 7 and 
Chap. 9: all the solutions of nonlinear equations presented in these chapters are 
obtained by solving the corresponding difference equations. 
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5.2.1 Consider the equation j„ = - J„_i + n. 

a) Use this equation to find j„ y2, y^, y^, Js, and jg in dependence on the 
initial value y^. 

b) Use these expressions to guess the solution j„, where « = 0, 1, 2, .... 

5.2.2 Consider the equation j„ = a y„_y + bn + c. Here, a, b, and c are constants, 
and n= 1,2, .... The solution of this equation is given by 



y„ = « Jo + 



\ — a 



1-fl 
n- a- 



\ — a 



1 " 
l-a 
+ c- 



1-fl 



a) Show that this solution is correct for « = 0. 

b) Show that this solution solves the equation y„ = a y„_^ +bn + c. 

5.2.3 Consider for the bracket term involved in exercise 5.2.2 the identity {n = 0, 

1,2,...) 

1 " 



n-l . n 11-1 , o /j-3 . . / 1 \ 1 . 

fl +zfl +ia H l-(«-l)fl+«fl 



1-fl 



n- a- 



1 - fl 



a) Show the correctness of this identity for « = 0, « = 1, and n = 2. 

b) Show the correctness of this identity for n = A and fl = 7. 

5.2.4 Consider the difference equation j,, = a >'„_i + bn + c and its solution given 
in exercise 5.2.2. Regarding the last term in the solution to this equation it 
is known that 

lim = lim(fl""' + a"'^ +--- + a +a']=n. 

"^1 1-fl fl^i 

a) Calculate the limit fl — > 1 of the identity given in exercise 5.2.3. 

b) Find the solution of the equation j,, = a y„_i + bn+ c for the case fl = 1. 
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5.2.5 Consider the equation y„ = a y„_x + bn + c and its solution given in exercise 
5.2.2. Assume that a, b, c, andj'o are nonzero. 

a) Calculate the asymptotic solution features for « — > co for the cases a <0, 
a = 0, and a > 0. 

b) There are observations that show an asymptotic behavior j„ = 2 e'". Your 
task is to choose model parameters a, b, c, and jo such that one trend of 
the solution of the equation y„ = a y„_x + bn + c agrees with the observa- 
tions. Explain the corresponding constraints for model parameters. 

5.2.6 Consider the compound interest formula ^n = (1 + pi r)" Sq. 

a) Consider two options to earn interest. A first option offers interest com- 
pounded monthly with an interest rate of 4%. A second option offers 
interest compounded annually with an interest rate of 100/»%. Which/? 
brings the same amount as given by the first option? 

b) Calculate the interest rate that doubles any initial principal in five years. 
The conversion period is three months. 

5.2.7 Consider the simple interest formula S„ = (1 + n p) Sq and the compound 
interest formula 5,, = (1 + p I r)" 5*0. There are three options to earn interest. 
Company A offers simple interest at a rate of 6%. Company B offers com- 
pound interest at a 4% rate with a conversion period of one month. Com- 
pany C offers compound interest at a 4% rate with a conversion period of 
three months. 

a) Calculate for the three cases the amount on deposit after 5, 10, 15, and 
20 years for any principal Sq. 

b) Which interest offer maximizes the amount on deposit after 5, 10, 15, 
and 20 years? 

5.2.8 The loan on a house is $200,000. 

a) Calculate the monthly repayment needed to have the loan repaid after 
30 years. The interest rate is 5%. 

b) Calculate the total amount paid back on the loan. 

5.2.9 A company deposits a sum of money 5*0 in a fund earning 100/?% interest 
compounded monthly. The company also deposits a sum Sq in this fund at 
the end of each conversion period. 

a) Find the difference equation for this problem and its solution. 

b) Simplify the solution for the case that/»/r « 1. 

5.2.10 A company deposits $1000 in a fund earning 5% interest compounded 
monthly. In addition, the company also deposits a sum D in this fund at the 
end of each conversion period. 



5.7 Exercises 203 



a) Find the difference equation for this problem and its solution. 

b) Which additional payment D is needed to have a balance of $30,000 
in the account after three years? 

5.3.1 Consider the second-order difference equation (h = 2, 3, . . .) 

y,. = --j„-2- 

The initial data are given by Jq = 2 and Ji = 0. 

a) Find the solution of this difference equation. 

b) Prove the validity of this solution. 

c) Determine the asymptotic solution for « — > oo. 

5.3.2 Find solutions for the following second-order difference equations, where 
« = 2, 3, . . .. The initial data for all cases are given by jo " 1 and ji = -1. 

«) J„ =2j„_2-l, 

b) y,, = y„-i - -y„-2 + -^ 

c)y„ =2;'„_i-2j„_2-l. 

5.3.3 Consider the second-order difference equation (« = 2, 3, . . .) 

The initial data are given by Jq = 1 and j'l = 0. 

a) Find a particular solution Y„ of this difference equation. Hint: consider 
the suitability of a quadratic function of n. 

b) Rewrite the problem as an equation for x„ = y„ - Y„. Hint: you have to 
provide initial data for x„. 

c) Find the solution x„ that satisfies the equation and initial data for x„. 

d) Show that the resulting solution for y„ satisfies both the equation for y,^ 
and the initial data for j„. 

5.3.4 In 1202 Fibonacci, a famous Italian mathematician who is known for the 
spreading of the Hindu-Arabic numeral system in Europe, was interested in 
the reproduction of rabbits. He considered the following conditions: 

• One male rabbit and one female rabbit have just been bom. 

• A rabbit will reach sexual maturity after one month. 

• The gestation period of a rabbit is one month. 

• A female rabbit gives birth every month after reaching sexual maturity. 

• A female rabbit will always give birth to one male and one female rabbit. 

• Rabbits never die. 
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a) Calculate the number of the pairs of rabbits for the first five months. 

b) Derive the difference equation that describes the number of the pairs of 
rabbits per month. 

c) Solve the difference equation. 

d) Calculate how many pairs of rabbits will there be a year from now. 

5.3.5 Consider a modification of the red blood cell production model discussed 
in Sect. 5.3.4. In an attempt to formulate a better model, we assume that the 
number of red blood cells produced by the bone marrow is a constant K 
(which means that M„_, in Eq. (5.80a) is replaced by K). 

a) Solve the resulting difference equation analytically. 

b) Assume that R^ = 0.9,/= 0.01, and K = 0.01. Discuss the suitability of 
the resulting solution as a model for the red blood cell production. 

5.3.6 Consider the model R„ = a R„_i + h R„_2 for the number of red blood cells. 
Here, a and b are any model parameters (i.e., we do not consider a = 1 -/ 
and b ^fas in Sect. 5.3.4). We assume that the number R„ of red blood 
cells can be described by the function R„ = A cos(a) n - 5), where A, o, and 
Jare model parameters. 

a) Calculate the parameters a and b so that R„ = A cos(o n- S) is a solution 
of the second-order difference equation 7?„ = a R„_i + b R„_2- Hint: use the 
relation cos(zi +Z2) = cosz, COSZ2 - sinzj sinzj. You should also use the 
relations sin 2 z = 2 sin z cos z and cos 2z = 2cos^z- 1. 

b) Use the results for a and b to explain the reason for the shortcomings of 
the expressions a = 1 -/and b =/used in Sect. 5.3.4. 

5.4.1 Consider the following nonlinear first-order difference equations, where n 
= 1, 2, .... Find the equilibrium points and determine whether the equilib- 
rium states are stable or not. 

, 1 



y„-i 



b)y., . , 
1 + y„-i 

c)y„ ^y„-i(^-y„J)- 

5.4.2 The Ricker equation y„ = ay„_i exp(-/3y„_i) is often used for the modeling 
of fish populations. In this equation, a is the maximal growth rate of the 
organism, and /^ represents the inhibition of growth due to overpopulation 
(Edelstein-Keshet 2005). 

a) Calculate the nonzero equilibrium solution to this equation. 

b) Show under which conditions the equilibrium solution is stable. 
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5.4.3 The Hassel model j„ = ay„_i (1 + y„-[y^, which may be seen as a limiting 
case of Ricker's model, has been used for the analysis of insect populations 
(Fulford et al. 1997). Here, a and y^ are positive constant. 

a) Calculate the nonzero equilibrium solution to this equation. 

b) Show under which conditions the equilibrium solution is stable. 

5.5.1 Consider the logistic difference equation}',, =>>„_! + ay„_i (1 -y„_i/ K) that 
was considered in Sect. 5.4.2: see Eq. (5.97). 

a) Determine the scaling of the model parameters with the time interval A^ 
so that the discrete derivative {y„ - y„_i) / At is independent of At. 

b) Present the differential equation for the continuous function y(t) that is 
related to the difference equation considered. 

5.5.2 Consider the equation j„ = ay„_i exp(-/?v„_,) considered in exercise 5.4.2. 
Assume that « = e^, where A is any parameter. 

a) Find the scaling of the model parameters A and /? with the time interval 
At such that the discrete derivative (y„ - y„_i) I At is independent of At in 
the hmit A? ^ 0. 

b) Present the differential equation for the continuous function y{t) that is 
related to the difference equation considered. 

5.5.3 Consider the population model y„ = ay„_i (1 + j„_i)"^ considered in exercise 
5.4.3. Assume that a = A^, where^ is any parameter. 

a) Find the scaling of the model parameters A and P with the time interval 
At such that the discrete derivative (y„ - y„_^ I At is independent of At in 
the hmit A? ^ 0. 

b) Present the differential equation for the continuous function y{t) that is 
related to the difference equation considered. 



6 Stochastic Changes 



The characteristic features of solutions of several types of difference equations 
were discussed in Chap. 5 to see which sort of processes can be described in this 
way. These discussions did not account for randomness. However, randomness 
effects are relevant to many applications. Examples are given by the diffusion of 
substances in the atmosphere or water, the chaotic motion of molecules and other 
particles in fluids, and the development of population densities in time. Therefore, 
the deterministic methods presented in Chap. 5 will be extended in this chapter by 
the inclusion of randomness effects. The relevance of the stochastic difference 
equations considered in the following is that these equations provide the solutions 
of stochastic differential equations and equations for PDFs, which are the basic 
equations for the modeling of the evolution of stochastic processes. Hence, this 
chapter provides the basis for the discussions in Chaps. 8 and 10. In particular, the 
basic ingredient of stochastic evolution equations (the Wiener process) and the 
basic methodology for the solution of stochastic evolution equations (Monte Carlo 
simulation) will be introduced in this chapter. 

This chapter is organized in analogy to the structure of Chap. 5. Section 6.1 
motivates the consideration of randomness in difference equations for changes. 
Sections 6.2 and 6.3 introduce linear stochastic equations: their general solution 
features and applications to diffusion processes will be discussed. Section 6.4 then 
carries this on into linear equations for changes with delays. The discussion of the 
diffusion model in Sect. 6.3 will be extended in this way by the discussion of the 
more general Brownian motion model. This latter involves the basic features of 
models for molecular motion - a starting point for the modeling of fluids (see 
Chap. 10). Section 6.5 then carries the discussion of linear stochastic changes in 
Sects. 6.2 and 6.3 to the domain of nonlinear stochastic changes. This section gen- 
eralizes the discussion of nonlinear population models in Sect. 5.4 by the inclu- 
sion of randomness. Section 6.6 summarizes the observations made in this chapter. 
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6,1 Motivation 

Diffusion. Let us consider first an example to illustrate the need to analyze the 
properties of stochastic difference equations. Many (or even most) processes in 
nature and technology are driven by (temperature, energy, velocity, concentration, 
...) changes. Such processes are called diffusion (or dispersion) processes because 
the quantity considered (e.g., temperature) is distributed until an equilibrium state 
is established (i.e., until the differences that drive the process are minimized). 
There are many examples of diffusion processes. Diffusion is responsible for the 
distribution of sugar throughout a cup of coffee. Diffusion is the mechanism by 
which oxygen moves into our cells. Diffusion is of fundamental importance in 
many disciplines of physics, chemistry, and biology: diffusion is relevant to the 
sintering process (powder metallurgy, production of ceramics), the chemical reac- 
tor design, catalyst design in the chemical industry, doping during the production 
of semiconductors, and the transport of necessary materials such as amino acids 
within biological cells. 

Diffusion Model. Let us consider a diffusion model to see the mathematical 
structure of such models. In particular, we consider the model (« = 1, 2, . . .) 

yn=yn-l+rS„_i. (6.1) 

Here, y„ refers to the position of any particle (for example, the height of any tracer 
above ground). The initial position y^, is assumed to be given. For simplicity, we 
assume that y^, is a deterministic parameter such that all the particles start at the 
same position. The random number Si. accounts for the effect of randomness (it 
models random up and down motions due to diffusion), and the parameter r is a 
deterministic parameter that modifies the intensity of randomness. The quantity ij, 
is assumed to be normally distributed. The specification of a normal PDF requires 
the definition of the mean and variance. Here, we assume that s^ is a standardized 
normal process with zero mean <Si> = and variance <£•/> = 1. These settings 
specify s^ at any k. In this way, nothing is said regarding the relation of Sf. and s„„ 
where k ^ m. At every step, the noise process is considered to provide a new input 
that is independent of previous noise values. Therefore, we assume that ^^ and s„ 
are independent random variables, <£i, s„> = 0, for k + in. These assumptions for Sj. 
can be summarized in the following way: 

(^*)-0, {s,s„,) = S,„. (6.2) 

Here, 4,„ refers to the Kronecker delta that is defined by the conditions Si,,,, = 1 for 
k = m and S,,,,, = for A: t^ m. The solution to the stochastic difference equation 
(6.1) can be found by having a look at the implications of this difference equation. 
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Fig. 6.1. An illustration of features of the diffusion model (6.1). Ten realizations 
of solutions are shown in (a), where n refers to the step number. The PDF/(y) of 
position values is shown in (b) and (c) at « = 10 and n = 40, respectively. 



Regarding the first three values y„ we find 

y\ =>'o + ''^o. 

yi =yi+rs, =>'o+r(^o+^i), 

^3 = J2 + '"^2 = Jo + '"(^0 + ^1 + ^2)- 



(6.3a) 
(6.3b) 
(6.3c) 



Here, we substituted y\^y(, + r e^ in jj, and J2 ^ J'o + '" (^'o + ^d in yv Therefore, we 
can conclude that the solution j„ is given by (« = 0, 1, . . .) 



y„ =yo + r{sa 



+ e, +£•,+■ 



■ + ^„-i)- 



(6.4) 



Usually, a problem is solved if the solution for any equation is found: the solution 
can be illustrated and the effect of parameter variations can be studied. Examples 
for the solution (6.4) are shown in Fig. 6.1, where j'q = 1 and r = 0.05 are applied. 
One realization is obtained by using one specific choice of random numbers in 
Eq. (6.4). It is evident that every realization of Eq. (6.4) is different: it is impos- 
sible to analyze the solution features in this way. The only way to analyze such 
solutions is to look at the statistics of this process at every n. In particular, the 
evolution of the PDF has to be determined in dependence on n. Examples for 
PDFs of position values are given in Figs. 6.1b-c. The PDFs that are shown in 
these figures represent normal PDFs with mean one and variance n i'^ (a detailed 
discussion of the properties of diffusion models is provided in Sect. 6.3). These 
PDFs reveal the nature of a diffusion process: the variance increases linearly with 
«, which simulates the distribution of the tracer considered in space. 

Processes Considered. There is a variety of stochastic processes, and it would 
be hardly possible to cover all these processes here. Therefore, we will focus in 
the following on processes with relatively little stochastic changes (i.e., diffusion 
processes). Such processes are relevant to a huge variety of processes, but there 
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are also cases for which the use of other concepts is more appropriate. One exam- 
ple for such a case is given by processes that jump from one integer to another 
integer (e.g., a population density that changes like 79, 81, 94, 103). Another 
example for such a case is given by processes that are bounded, for example by 
zero and one, according to their definition (e.g., mass fractions of chemical spe- 
cies). The modeling of such processes in terms of a normally distributed noise can 
imply unphysical negative values of variables. 

Questions Considered. The illustration of the modeling of diffusion processes 
in terms of Eq. (6.1) leads to questions like the following ones: 

• What is an appropriate noise model, and what are its characteristic properties? 

• How does a noise model affect the statistical properties of a process considered? 

• How can we evaluate the suitability of a model for a stochastic process? 

These and other questions will be discussed in the following sections with regard 
to several relevant diffiision processes in mechanics and population ecology. 



6.2 Linear Stochastic Changes 

Let us discuss first an extension of Eq. (6.1): we consider for a variable y„ the 
first-order linear difference equation (« = 1, 2, . . .) 

y„=ay„^i+b + rs„_i. (6.5) 

Here, a, b, and r are any deterministic model parameters. We assume that y„ has a 
known normally distributed initial value jo ^t n = 0. The noise term £/. is assumed 
to be normally distributed with mean zero and variance one. For k ^ m, Sf. and £,„ 
are independent random variables. Therefore, £i. is characterized by the properties 

(^/.) = 0, {e,e,„) = S,„. (6.6) 

It should be mentioned that these assumptions are sufficiently general: any non- 
zero mean value of £i can be combined with h, and any variance of £), unequal to 
one can be combined with r. We also have to specify the correlation between S/^ 
and Jo because y^ is a random variable. The noise model should be independent of 
the initial distribution. Thus, we assume here that e^. and yg represent independent 
random variables. In the following we will derive and illustrate first the analytical 
consequences of Eq. (6.5) for the one-point statistics (the properties of j^ at one k) 
and correlations (between values yi, and y,„ with k ^ m). Second, we will consider 
the typical features of numerical solutions of Eq. (6.5), i.e., we discuss the proper- 
ties of Monte Carlo simulations. The results obtained in this section will be used 
in the following sections of this chapter. 
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6.2.1 One-Point Statistics 

Solution. In accord with the derivation of the solution of a deterministic linear 
first-order difference equation in Chap. 5, let us consider first the implications of 
Eq. (6.5) for n = 1, 2, and 3, 

yi^ay^+b + rSf^, (6.7a) 

jj = ay^+b + re^ - a{ay^+b + r£^^) + b + re^ 

= a yg + a{b + i'£f^) + b + i'Si, 
y^ = ay2 +b + rs^ - a(a^y^ + a (b + r e^) + b + r £^) + b + r £2 

= ayQ+a (b + r£f^) + a(b + i'£i) + b + i'£2. 



(6.7b) 
(6.7c) 



Here, the expressions for previous values of j',, are used for representing the next 
y„ in terms of yQ. By following these patterns we can find the solution j„ of Eq. 
(6.5) for« = 1, 2, ..., 

j„ = a"y, + a'-' (b + r£,) + --- + a' (b + r £„_2 ) + a°(b + r £•„_! ) . (6.8) 

This solution generalizes the solution (6.4), which is obtained for a = 1 and b = Q, 
and it generahzes the solution (5.9), which is obtained by neglecting £1^. As illus- 
trated in the introduction, the knowledge of such a random analytical solution is 
not directly helpful. However, this analytical solution can be used for deriving 
statistics of the stochastic process considered. 

Moments. First, let us have a look at low-order moments of j„ that are implied 
by Eq. (6.8). The mean value <y> can be obtained by averaging Eq. (6.8). Due to 
the fact that £1^ has a zero mean, we obtain 

{y„) = a"{y,) + b{a"-'+- + a'+a'). (6.9) 

This expression is equal to the solution (5.9) of the corresponding deterministic 
difference equation. We use the identity (5.10), 

1 " 

a +a +--- + a +a = , (6.10) 

1-fl 

to write the mean value more conveniently, 

{y„) = ci"{y,) + b^-^. (6.11) 

1-fl 

To derive a corresponding expression for the variance it is helpful to use Eq. (6.9) 
for deriving an expression for the fluctuating solution y ^ y„ - <y„>- By taking 
the difference between Eqs. (6.8) and (6.9) we obtain 

yn-^ yo'^^y^ 6^0+^ £:,+■■■ + « e^^_2+a £„_]). (6.12) 
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The consistency of this expression may be seen by averaging it - which resuhs in 
zero on both sides. Equation (6.12) can be used for calculating the variance of >>„. 
By squaring Eq. (6.12) and taking the average we find 

/~ 2\ lnl~2\ , 1, 2(;i-l) , 2(;i-2) , , 2 , On /^ 1 T-, 

{y,, ) = '^ \>^o ) + '" (fl + a ' +■■■ + a + a ). (6.13) 

The latter expression makes use of the fact that Sf; and s„, are independent random 
variables for k-^ m, and that £-t and y^ are also independent. Hence, only averaged 
squared terms of the contributions in Eq. (6.12) contribute to the variance formula 
(6.13). Equation (6.13) can be rewritten in correspondence to Eq. (6.11) for the 
mean by using Eq. (6. 10) for a', 

(fl')"-' + («')"-' + ■■■ + (a')' + {ay = V-^- (6-14) 



Correspondingly, we obtain 

5^;) = «"(7o^) + '-^. (6.15) 

Higher-order moments can be calculated in the same way. Due to the property of 
Jo and Sj; to be normally distributed it is not hard to see that the odd moments of 
y disappear. The calculation of even moments of y in this way is relatively 
complicated. 

PDF. A convenient way to completely determine the one-point statistics im- 
plied by Eq. (6.5) is to calculate the PDF ofy„. The latter can be done on the basis 
of the theorem (10.75), 

X, ~ ^(//,,a/) => £x,,~^f£//,,£a,0. (6.16) 



This theorem states the following: We consider random numbers X^ (i ^ 1, A') that 
are normally distributed with mean //, and variance af'. Then, the sum of J:^ values 
is also normally distributed. The mean of this normal PDF is the sum of all //,, and 
the variance of this normal PDF is the sum of all variances ai^. For our case, the 
sum of J:^ values is given by the terms in Eq. (6.8). Due to the properties of jo and 
£i, all single contributions are normally distributed. Thus, the PDF of >>„ also has to 
be normally distributed. The mean of the PDF of j',, is given by Eq. (6.9), which 
represents the sum of the means of all contributions. The variance of the PDF of j„ 
is given by Eq. (6.13), which represents the sum of the variances of all contri- 
butions. These observations are summarized by the formula 






i}'-{yS 



2{y: 



(6.17) 
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for the PDF/,(y) of j',,. Here, y is the sample space variable corresponding to the 
random variable y„. The mean and variance of this PDF are given by Eqs. (6.11) 
and (6.15), respectively. The notation /,(>>) refers to the dependence of n, this 
means the PDF (6.17) evolves with n. Equation (6.17) can be applied for finding 
higher-order moments by using the properties of normal distributions discussed in 
Sect. 4.3.2. 



6.2.2 Correlations 

Correlation Relevance. A basic characterization of a stochastic state is given 
by the mean and standard deviation, which explain the typical value and range of 
variations of any random variable considered. The consideration of the evolution 
of a stochastic process leads to the additional question about the typical lifetime of 
fluctuations: to model the dynamics of fluctuations in terms of a stochastic model 
we need to know after which characteristic time (one nanosecond, or one second, 
or one hour?) fluctuations will disappear. Information about the typical lifetime of 
fluctuations is available via the correlation <y y >■ This quantity is considered 
at a fixed n as a function of m = 0, 1, 2, .... Usually, we consider the normalized 
correlation function 

r^ i \ \ynyn+mj / r ■, o\ 

CXm) = ^j::^T^, (6.18) 

\ynyn) 

which is equal to the correlation coefficient between y„ and y„, if the variance is 
stationary (if < j j > ^ <>' y >). It is often found that C„(m) is a decreasing 
function of m. By analyzing at which m the correlation function C„(w) is signif- 
icantly reduced (e.g., below 1%), we know the typical lifefime of fluctuations. 

Correlation Calculation. Let us calculate the correlation < v v > for the 
process j„ determined by Eq. (6.5). The simplest way to perform this calculation is 
to consider the pattern of these correlations. This approach applies the difference 
equation for fluctuations, which is obtained by taking the difference between 
Eq. (6.5) for j„ and the implied equation for the mean value <y„>, 

y„ =yn-{yn) = ay„_i+h + r£„_i-[a{y„_,) + b)=ay„_i+rs„_,. (6.19) 

We consider this y^ atn+l, multiply it by y^^ and average this expression, 

{y„y„+ 1 ) = {y„ (« J„ +''S„)) = a{yj„ ) + r{y„s„ ). (6.20) 

The last term here can be evaluated by using Eq. (6.12) for y . Due to the fact 
that £•„ is independent of all its previous values and Jq, we find that the last term in 
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Fig. 6.2. An illustration of the variance (6.15) in dependence on n. Here, j'o = 
and r = 0.8 are applied. The model parameter a applied is given in the plots. 



Eq. (6.20) has to disappear, < y^^s^^ > = 0. Hence, < j„ j,,^, > is given by 



(6.21) 

The corresponding features for < >'„>'„^2 '^ ^'^^ ^^ found correspondingly. First, we 
use Eq. (6. 19) for the representation 



{y„y„+2 ) = {y„ (« ?„+ 1 + '" ^„+i )) = a{y„y„+ 1 ) + r(y„£„+ 1 ). 



(6.22) 



The last term has to disappear: the reasoning is the same as that used for showing 
that <y£ > = 0. By means of Eq. (6.21) we find 



{y„y„+2) = a {y„y„ 



(6.23) 



By using the same approach for deriving corresponding expressions for < y^y^^^ > 
and so on, we find the general expression for correlations to be given by 



\y„yn+ 



a [y„y„ , 



(6.24) 



where m = 0, 1, 2, .... The setting m = results in an identity for the variance of 
y„. The normahzed correlation function C„(m) defined by Eq. (6. 18) is given by 

C„im) = a"'. (6.25) 

It should be pointed out that C„(m) is independent ofn for the process considered. 

6.2.3 Solution Features 



Let us illustrate the typical statistical properties of solutions to the difference 
equation (6.5). We will consider here the means and variances that determine the 
normal distribution (6.17) and the normalized correlation function C„{m). 
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Fig. 6.3. An illustration of realizations of solutions of the difference equation 
(6.26) in dependence on n. Here, a = 0.5, fo = 1, r = 0.8, and>'o = are applied. 



Means and Variances. Equation (6.11) for <y„> agrees with the solution (5. 15) 
of the corresponding deterministic equation (5.6) where Sf- = 0. Therefore, <y> is 
characterized by the features of the deterministic j„ that are illustrated in Fig. 5.1. 
Figure 6.2 illustrates that the variance shows a very similar behavior as the mean 
<y„> (there are no curves corresponding to the curves with negative values of a in 
Fig. 5.1 because the variance depends on a^). The reason for that is the very 
similar structure of formulas: instead of a, b, and <ya> in the mean formula (6.11) 
we have a^, /"^ and <>■„" > in the variance formula (6.15). Regarding the modeling 
of processes, the most relevant case is given if \a\ < 1 such that an equilibrium 
state is achieved. The corresponding equilibrium values (for large values of n) of 
means (6.11) and variances (6.15) are given by <y> = Z>/(1 -a) and <y^>^i'^/ 
(1 -a^). These relations can be used for the determination of the model parameters 
b and r^ by using known equilibrium values for the mean and variance. The model 
parameter a has to be chosen such that the transition from the initial value to the 
equilibrium value is represented as good as possible. We note that the equilibrium 
variance is larger than the variance i'^ of the noise process r s^-. 

Correlations. The normalized correlation function (6.25) reveals the following 
features: First, usually it does not make sense to consider negative parameter a 
values: C„{m) will oscillate then similar as the mean values in the lower plots of 
Fig. 5.1. The modeling of correlations in this way is not helpful for applications. 
For values a > 1, C„{m) will increase. Such a behavior is unreasonable because 
there is usually no mechanism that can increase correlations. For a = 1 we find 
C„(m) = 1, which means that the correlation will never change. Such a case was 
considered in the introduction: see Eq. (6.1). Such a correlation behavior is help- 
ful for the modeling of real processes if there is no mechanism that destroys corre- 
lations. The last case is given if < a < 1 . The correlation function C„{m) will 
decay with n in this case. This is the common behavior of correlations: memory is 
usually lost. This discussion reveals that the correlation function shows a mean- 
ingful behavior if the condition < a < 1 is satisfied. 
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6.2.4 Monte Carlo Simulation 

Example. The resuhs obtained in Sects. 6.2.1 and 6.2.2 lead to the question of 
how such conclusions can be validated. One way of doing this is to confirm such 
findings on the basis of simulations, i.e., by an analysis of simulation results. 
Analytical conclusions as those obtained in Sects. 6.2.1 and 6.2.2 are often un- 
available. In this case, the only way of obtaining insight into the statistics of 
processes is the analysis of simulation results. The process of doing this will be 
illustrated in this section. We consider again Eq. (6.5), 

>"„ =«J„-i+^ + '"^„-i> (6-26) 

but we work now with specified model parameter values: we assume a = 0.5, 
b=\, r = 0.8, and jo = 0. Several realizations of Eq. (6.26) are illustrated in 
Fig. 6.3. Equation (6.26) is only used here for the generation of simulation results 
that can be analyzed. The analysis of the evolution of the mean and variance, the 
PDF and normalized correlation function will be described in the following. 

One-Point Statistics. The mean <y>, standard deviation < y ^ >"^ and PDF 
/,(>>) can be calculated at every n as described in Sect. 4.4.1. The numerical results 
are shown in Fig. 6.4 in comparison to the corresponding analytical results pre- 
sented in Sect. 6.2.1. It may be seen that there is no observable difference between 
the numerical and theoretical results for the 10' realizations applied (the effect of 
variations of the number of realizations was already described in Sect. 4.4.1). The 
evolution of the PDF with n is illustrated by two examples: the PDF/i(y) at « = 1 
and the PDF/io(y) at « = 10. In consistency with the evolution of <v„> shown in 
Fig. 6.4a, Figs. 6.4c-d show the variation of the mean value <y> (which is equal 
here to the PDF maximum position) with n: we have <y{> = 1 and <Jio> ^ 2. In 
correspondence to Fig. 6.4b, the standard deviation increases with «: we have 
< y; >"' = 0.80 and < y^^^ >"' = 0.92. 

Correlations. The normalized correlation function C„{m) can be calculated 
numerically in the following way (A' refers to the number of realizations applied 
andy=l,AO: 

1. Use the difference equation (6.26) to find j'„(/), where « is a fixed value. Store: 

•(v„) •y.XJ) = ySJ)-{y„) •(?„') (6.27a) 

2. Continue with the solution of the difference equation (6.26) to find y„+,„{j) for 
varying m values. Calculate for each m=0, 1, 2, ...: 

\y n+m I y n+m\J y ~ y n+m^J y \y ii+m / 

^,, {y„y„.„,) 1 i^~,.,~ ,., (6.27b) 

• C„(m) = ' ^ ' = -j:—r—z.y„U>y„+„Xj) 
{y„y„) {y,')NM 
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Fig. 6.4. The analysis of Monte Carlo simulation results. The solid lines in (a) and 
(b) show the mean value and standard deviation of j„ obtained by Monte Carlo 
simulation in dependence on n. The solid lines in (c) and (d) show the PDF/i at 
n= \ and the PDF/io at n = 10 obtained by Monte Carlo simulation, respectively. 
10' realizations and a filter width Ay = 0.1 were applied to calculate these PDFs. 
The corresponding analytical results are shown by dashed lines in all these figures 
(there is no observable difference). 
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Fig. 6.5. The analysis of Monte Carlo simulation results. The solid lines in (a) and 
(b) show the normalized correlation functions Cj at « = 1 and Cjo at « = 10 in 
dependence on m, respectively. 10' realizations were applied for this calculation. 
The corresponding theoretical result (6.25) is given in both plots by a dashed line 
(there is no observable difference). 
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The calculation of the normalized correlation function C„{tn) has to be performed 
carefully. For example, the fluctuations y,, (y) and y„+^ (J) have to be calculated 
in terms of the correct mean values <y> and <j„+„>, respectively. It is relevant 
that the correlation function is calculated as mean value of y„ (J) y„+,„ {J) , where 
y„ U) and y„+,„ (J) denote the values of one realization at different positions. The 
correlation functions Ci(m) at « = 1 and Cio(»?) at n = 10 are shown in Fig. 6.5 in 
comparison to the corresponding analytical results. As found with regard to the 
one-point statistics, there is no observable difference between the numerical and 
analytical results. The conclusion of Eq. (6.25) that C„(m) is independent of n is 
also confirmed by Fig. 6.5: there is no observable difference between the C„(?w) 
curves in both plots. 



6.3 Diffusion 

Let us consider random walk models (which are also called drunkard's walk 
models) to illustrate the application of the linear stochastic first-order difference 
equation discussed in Sect. 6.2. A random walk refers to a trajectory that results 
from taking successive random steps. Random walk models represent the basis for 
diffusion models that are applied in many areas (Durbin 1983): see Sect. 6.3.3. 
From a mathematical point of view, the focus of this section is on the introduction 
of the basic ingredient of stochastic evolution equations: the Wiener process. In 
the following we will analyze the features of the random walk model (6.1), 

y„=y,.-i+r£„-i- (6.28) 

Here, y„ is seen as a position. The random variable £„_i is normality distributed and 
characterized by <£,> = and <£i, £,> = 4»r We will also assume that s^ is 
independent of the random initial position Jq. 



6.3.1 Random Walk Model 

One-Point Statistics. The properties of solutions to Eq. (6.28) can be seen by 
specifying the results for the more general model (6.5): we have to set a = 1 and 
b = 0. The discussion in Sect. 6.2 showed that y„ is normally distributed. Accord- 
ing to Eq. (6.9), the mean of >>„ equals the mean initial value for the case a = 1 and 
b^O considered, 

{y„) = {yo)- (6.29) 
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Equation (6.13) shows that the variance ofy„ is given by 

y,')-(yo') + nr'. (6.30) 



Hence, y„ is nonnally distributed according to 

y„~^({yo),{n) + nr'). (6.31) 

The corresponding features of statistics are illustrated in Fig. 6. 1 : the mean value 
is constant, and the variance increases with n. 

Correlations. The correlations ofy„ are determined according to Eq. (6.25) by 

{y„y„+,„) = {y„y„), (6-32) 

where m = 0, 1, 2, .... The meaning of this result can be seen in a better way by 
writing this relation 

{y„iy„»,-y,.)) = 0- (6-33) 

This means that the change y -y is uncorrelated to y . The validity of this 
conclusion can be seen by proving that J'„+,„ - y„ is caused by noise contributions 
that are uncorrelated to y (see exercise 6.3.1). The steady addition of uncorrelat- 
ed noise contributions then implies the steady increase of the variance ofy„. 

Time Dependence. A closer look at the variance expression (6.30) reveals the 
following. We may use Eq. (6.28) as a model for a continuous diffusion in time 
n At. For this case, the variance (6.30) should be a function of n At. The latter is 
the case if we parameterize the noise coefficient r by 



r = ^DAt. (6.34) 

It is relevant to note that we do not have to assume here and in the following that 
At — > 0. The model parameter D represents a non-negative variable that is called 
the diffusion coefficient. In terms of the relation r = {D A?)"* we can write the 
variance < y^^ > ^ <y^^> + D n At. Therefore, D is the derivative of the variance 
by time n At. This fact explains why D is called the diffusion coefficient: D deter- 
mines the increase of the position variance (which describes the spreading of a 
plume). By using r = (Z) A?)"^, we can write the diffusion model (6.28) as 

y„=y„_,+^te„_,. (6.35) 

According to (6.31), y„ is normally distributed, 

y„~^[{y,),{yo') + DnAt). (6.36) 

Equation (6.32) implies for the normalized correlation function C„(m) that 

C„(m) = l. (6.37) 
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6.3.2 The Wiener Process 

Model Reformulation. The noise term in Eq. (6.35) involves three compo- 
nents: the random variable £•„_,, the scaling parameter (A?)"", and the diffusion 
coefficient D. The diffusion coefficient will change with the problem considered, 
but the first two components are the same for many kinds of problems. Thus, it is 
helpful to split the writing of the noise term in Eq. (6.35) into one contribution 
that represents the properly scaled noise, and another contribution given by the 
case-dependent diffusion coefficient D. The latter can be achieved by defining a 
variable W„ by the difference equation of j„ where D^\, 

W„=W„_,+^s,._, (6.38) 

To simplify this relation we introduce the abbreviation AW„_i = W„ - ff„_„ such 
thatEq. (6.38) reads 



AW„_,=^Ate„_,. (6.39) 

By making use of this definition of AW„_i we can write the model (6.35) as 

y,,=y„^,+4DAW„_,. (6.40) 

This writing represents the standard formulation of stochastic difference and dif- 
ferential equations (see Chaps. 8 and 10). The noise model is written in terms of 
the case-independent scaled random variable AW„_i, which enables the calculation 
of statistics that are functions of time t = n At, and the case-dependent diffusion 
coefficient D. 

Wiener Process. The stochastic process W„ is called the Wiener process. 
It represents the most important stochastic process. Thus, let us have a closer look 
at its properties. The comparison of Eq. (6.38) for W„ with Eq. (6.35) forj,, shows 
that statistical properties of W„ can be obtained by setting Z) = 1 in the correspond- 
ing statistics of y„. Therefore, we find that the Wiener process W„ is normally 
distributed according to 

W„-^{lW,),{w,') + nAt\ (6.41) 

As found for j,„ the correlations of W„ are determined by 

{W„W„.„)-{W„W,). (6.42) 

Wiener Process Changes. In addition to the properties of W„, it is helpful for 
the discussions below to consider the properties of the change AW„ of a Wiener 
process. The definition AW„ = (A?)"^ £„ shows that AW„ is normally distributed, 

AW,, ~ 5V(0, At). (6.43) 
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The correlation properties of AW„ also follow from AW„ = (At) £•„, 

, , I A^ if « = 7w 

{AW„AW,„) = \ , (6.44) 

where we made use of <Si^ s,> = 4m- We apphed AW,^ ^ AW,,, which is the same 
because o{<AW„> = 0. By dividing both sides by (AtY we can write 



^W„ AW„, 
At At 



1 -f 

— nn = m 

At . (6.45) 

if « 5^ m 



This result means that the variance of the derivative AW„I At of W„ does not exist 
for A? — > 0: the variance goes to infinity. Consequently, W„ is not differentiable 
because the probability for the appearance of values of AW„ I At that are larger 
than any limit is equal to one (Gardiner 1983). 



6.3.3 Diffusion Models 

How is it possible to use the random walk model (6.28) as a diffusion model 
(i.e., a model for the transport of any inert substance due to the random motions of 
any fluid)? In particular, how can we calculate the evolution of the concentration 
of any inert substance? Let us address these questions in the following. 

Definition of Concentration. First, we have to define the mean concentration 
C of any substance considered. The general relation of the mean concentration of 
a substance to the statistics of random trajectories can be found elsewhere (Durbin 
1983, Seinfeld & Pandis 2006). Here, we will focus on a simple but relevant case: 
we consider an instantaneous emission from a point source, i.e., the emission of a 
mass M at time zero at a fixed position jj. For this case, the mean concentration C 
is given by M times the VDV f„(y) for finding a parcel at a step n at a position j, 

C„=Mf„{y). (6.46) 

By using the result (6.36) for the PDF of >>„ we find the mean concentration to be 
given by 

C„=^^exp|-l^^|. (6.47) 

Here, the mean value is equal to the fixed position y^, and the variance is given by 
a,^ = D n At. Equation (6.47) describes the temporal evolution of the mean con- 
centration in one dimension: the j-axis. It is worth noting that the total mass 
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involved remains constant for all n, 

jC„dy=M, (6.48) 

as may be seen by using the normalization property of the normal PDF. 

Initial Condition. Is Eq. (6.47) consistent with the assumed initial condition? 
The direct calculation of the initial concentration Cq according to Eq. (6.47) is 
problematic because of a^^ = 0. The best way to calculate the initial concentration 
is to consider the limit a„^ — > 0. According to Eq. (6.47), the initial concentration 
at is then given by 

C,=MS{y-y,), (6.49) 

where we use the expression 

Siy-y,)=\im^=^cxp\- ^y~y'} . (6.50) 

This function is another representation of the delta function, see the discussion in 
Sect. 4.2.2 and exercise 6.3.3. This function is a normal PDF centered at y^ with 
vanishing variance. This means that the only nonzero concentration is found at 
y = Jo- Due to the fact that the integral of Cq over y is equal to M, we find that all 
the mass M is emitted at j = yg, as assumed in the preceding paragraph. 

Boundary Effects. The concentration formula (6.47) assumes that there are no 
boundaries, which is not the usual case if we consider diffusion processes. Let us 
consider two relatively simple examples to show how the effect of boundaries can 
be accounted for. A first case is given if there is a total reflection of material at 
y = 0. The presence of such a totally reflecting boundary can be taken into account 
by assuming that there is a hypothetical source at j = -jvq. The contributions of the 
sources aXy^yg and y = -y^ then result in the concentration 



To check the validity of this formula, let us integrate C„ over the range < j < co, 
= -j=< J e ar - j e as> = —j=< J e as + je as> (6.52) 

[-I'o'v-""'! -I'o'V^CT,, J I -J'o'V^CT,, J 

= —j= ]e ds = M. 

yjK -00 
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Here, we introduced the variables r = (y - Jo) / (2"^ cr,,) and s = -(y +>'o) / (2"^ o;,). 
The last expression makes use of Eq. (4.70). Hence, all the material is conserved 
for this case of a totally reflecting boundary. As a second example, let us consider 
the case of a totally absorbing boundary aty = 0. The presence of such a boundary 
can be accounted for by assuming that there is a hypothetical source at j = -y^. 
The difference to the approach applied to account for a totally reflecting boimdary 
is that we have to consider now the difference of both distributions to ensure that 
C„ = at the boundary y^O, 

To find out how much material is left in the accessible domain < >> < co for this 
case we integrate again C„, which leads to 



jC„dy = -^ \ e-''ds. (6.54) 



The proof of this formula is the concern of exercise 6.3.6. As expected, the total 
amount of material is not conserved in general for this case because the integral is 
smaller than ;r"^. It is worth noting that C„ integrates to M if Jq/ (2"^ o;,) — > oo. 

Ground Concentrations. The mean concentration development in time t = n/S.t 
can be used to find a corresponding two-dimensional concentration in a x-y plane, 
where x and y refer to the horizontal and vertical coordinates. The latter can be 
achieved by assuming that the substance is transported along the x-direction with a 
constant velocity U = x^J (n At). By using the relation iiAt = x„ / U, the variance 
a^ = D n At = D x„ / U becomes a function of x„. Let us calculate the ground 
concentration at >> = for the case without boundary to illustrate the use of this 
approach. The results to be obtained can be applied to the case of a totally 
reflecting boundary by multiplying the ground concentration for the case without 
boundary with two. The consideration of the ground concentration for the totally 
absorbing boundary case does not make sense because this ground concentration 
is equal to zero. According to Eq. (6.47) and cr„" = DxJ U, the ground concentra- 
tion for the case without boundary is given by 



It is convenient to introduce the nondimensional positions x,„ = Z) x„ / (C/jo^) ^nd 
concentrations C.„ = C„yQ/M. Then, Eq. (6.55) can be written as 

V2;rx.„ [ 2x.„J 
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Fig. 6.6. The normalized ground concentration C,„ 
along the normalized horizontal coordinate x,„ for 
the case without boundary (see Eq. (6.56)). 



An illustration of the normalized ground concentration distribution is given in 
Fig. 6.6. It is reasonable that the C.„ curve shows a maximum. The plume does 
hardly hit the ground for very small distances x.,„ and the concentration C.„ has to 
become smaller for growing large distances x,„ because of the ongoing substance 
diffusion. The maximum position can be calculated by considering the first two 
derivatives of Eq. (6.56). We find for this case 
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(6.57) 
The first-order derivative disappears for x,„ = 1, and the second-order derivative is 
negative at x,„ = 1 . Consequently, the concentration has a maximum at x,„ = 1 . The 
maximum value of C,„ is given by 



C, 



1 



2;r ^jlTie 



= 0.242. 



(6.58) 



Such analytical results give a valuable guideline for the evaluation of maximum 
ground concentrations depending on the nature of boundary. 






Fig. 6.7. Three examples for two-dimensional Brownian motion of particles. 
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6.4 Brownian Motion 

Next, let us consider the modeling of Brownian motion. Brownian motion of 
particles refers to the following phenomenon: fine particles, when suspended in 
water, move in an irregular way (see, e.g., the illustration in Fig. 6.7). In 1827, the 
botanist Robert Brown investigated such motions by considering the suspension of 
small pollen grains in water (therefore, all such irregular particle motions are 
called Brownian motion). Brown was interested to understand whether the chaotic 
motion of pollen grains is a manifestation of life. By repeating the experiment 
with particles of dust, he was able to rule out that the motion was due to pollen 
particles being alive, although the origin of the motion was yet to be explained. 
The relevance of this problem arises from the fact that the modeling of Brownian 
motion provides the basis for the development of models for molecular and fluid 
flow motion. However, the analysis of such models requires knowledge about the 
treatment of joint PDFs and joint PDF transport equations and their consequences. 
Therefore, such molecular and fluid motion models will be considered in Chap. 10 
after introducing the required mathematical concepts. Here, we will consider this 
problem because of two reasons. First, this discussion follows the approach used 
in Chap. 5 to continue after the discussion of linear first-order equations with the 
discussion of second-order difference equations. The second reason for consider- 
ing this problem is that the Brownian motion model represents an extension of the 
diffusion model discussed in Sect. 6.3. 



6.4.1 Brownian Motion Model 

Brownian Motion Model. The explanation for the Brownian motion problem 
was published by Einstein (1905). Einstein explained that the irregular motion of 
pollen grains is caused by the exceedingly frequent impacts on the pollen grain of 
the incessantly moving molecules of liquid in which it is suspended. Einstein 
developed a mathematical model (a Fokker-Planck equation: see the discussion in 
Chap. 10) for the explanation of Brownian motions. A few years after Einstein's 
explanation, Langevin (1908) presented the corresponding stochastic equations for 
the Brownian motion of particles. We will follow here Langevin's approach by 
considering the following stochastic difference equation system 

(6.59a) 
(6.59b) 



^,,-^,,-1 


- = ^,,-1, 




At 




V,. - V,,-, 


If 
- = — v„_, 


At 


At 
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Here, x„ and v„ refer to the position and velocity of a Brownian particle, respec- 
tively. No assumption is made regarding the time interval A?. D denotes again a 
diffusion coefficient, r represents a characteristic time scale, and the change of the 
Wiener process is defined by AW„_i = (A?)"^ £■„_[. The last term on the right-hand 
side of Eq. (6.59b) provides a random input (as a model for the random impacts of 
water molecules on a pollen grain), and the first term on the right-hand side of 
Eq. (6.59b) models the relaxation of the pollen velocity due to the damping influ- 
ence of surrounding water molecules: for a positive (negative) velocity v„_i there 
will be a decrease (increase) of the velocity due to this relaxation term. 

Linear Second-Order Difference Equation. Let us show first that the model 
(6.59) extends the discussion of a linear first-order difference equation in Sect. 6.3 
by the consideration of a linear second-order difference equation. Equation (6.59b) 
can be also written as 

l--]v„_,+-^lDAW„_,=(l--]^^^^ + -^AW„_„ (6.60) 

T J T \ T ) At T 

where Eq. (6.59a) was used to replace i;„_, on the right-hand side. The use of this 
equation at « - 1 on the right-hand side of Eq. (6.59a) results in 

l_A^^5izLZ5izl + lV^Af^,_,. (6.61) 



At y T J At T 

This equation represents a linear second-order difference equation for x„, 

x„ = x„_i + ( 1 - y 1 (^,,-1 - x„-2 ) + -y ^DAt s„_^ . (6.62) 

Here, the noise term is written by making use of AW„_2 = (A?)"^ s„_2- 

Comparison with Diffusion Model. Let us consider the difference between 
Eq. (6.62) and the diffusion model (6.28). We can write Eq. (6.28) in terms of x„ 



x„=x„_,+,[DAt£„_„ (6.63) 

where r = (Z) At)"^ is used. The random variable s„_2 can be replaced by s„_i, which 
does not make a difference. It may be seen that the setting At ^ r reduces the 
Brownian motion model (6.62) to the diffusion model (6.63), this means the 
diffusion model represents a coarse Brownian motion model in which the time 
step At is equal to the typical relaxation time r. The relevant difference between 
both models is given by the appearance of the second term on the right-hand side 
of Eq. (6.62). What is the advantage of involving this term? The consideration of 
this term is equivalent to the consideration of the first term on the right-hand side 
of Eq. (6.60). Without involving this term, the velocity v„ is proportional to 
AW„_i/At. It was shown in Sect. 6.3.2 that the variance of AW„_i/At does not exist 
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for a continuous diffusion process (i.e., for A? — > 0). Apart from that, the diffiision 
velocities would be uncorrelated due to the properties ofAW,,. Thus, the neglect of 
the first term on the right-hand side of Eq. (6.60) does not result in a sound model 
for diffusion in reality. In contrast to that, the consideration of this first term 
enables the modeling of correlated velocities that have a finite variance. 



6.4.2 Discrete Brownian Motion Statistics 

Joint PDF. Let us consider the consequences of the Brownian motion model 
(6.59). By adopting the same reasoning as with regard to y„ in Sect. 6.2, it is 
possible to show that both x„ and v„ are normally distributed if Xq and Vq are 
normally distributed as assumed here. However, this does not mean that the joint 
process (x„, v„) is also normally distributed. To clarify this question, we have to 
ask whether all the consequences (10.43) of a bivariate normal PDF for central 
moments of third-order, fourth-order, fifth-order, and sixth-order are satisfied (see 
the corresponding discussion in Chap. 10). By solving Eq. (6.59) for a number of 
10' realizations, one finds that the magnitude of the deviations between the calcu- 
lated and theoretical central moments of third-order, fourth-order, fifth-order, and 
sixth-order is smaller than 0.0085 for all the 22 conditions considered. The rela- 
tive error can be calculated for the nonzero fourth-order and sixth-order moments. 
This calculation shows that the magnitude of the relative error is smaller than 
0.22%. Therefore, we find in this way evidence that the joint process (x„, v„) is 
indeed normally distributed. 

Solutions. The best way to find the means and variances of the joint normal 
PDF of x„ and v„ is to calculate the solutions of Eq. (6.59). For doing this we use 
the abbreviations a = I -At/ rand rg = {D Atl z^)"", where I'g is the noise intensity 
in the Brownian motion velocity equation. Equations (6.59) read then 

x„ =J^„_i +Af [;„_,, (6.64a) 

t;„ = fli^„_i + 'g f„_,. (6.64b) 

For the first three i;,, we find according to Eq. (6.64b) 

f, =fli;„ +rg £„, (6.65a) 

fj = a f] + ^g f , = o'Vq + i-g (fl f „ + f 1 ), (6.65b) 

v^ = av2+rg £2 = o^Vq + r^ [cP' £f^+ a£^+ e^ j. (6.65c) 

Correspondingly, the solution of Eq. (6.64b) is given by the expression 

v„ = a" v„ + Fg (a""' £„+■■■ + a £„_2 + a° £„_i j. (6.66) 
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The solution for x„ can be similarly obtained by considering the implications of 
Eq. (6.64a) for « = 1,2, and 3, 

Xj = Xq+AIVq, (6.67a) 

X2= Xj+ Atv^ = Xg+ At{vQ+Vi), (6.67b) 

Xj = X2 +AtV2 = Xg +At(Vg +Vi +^2)- (6.67c) 
By following this pattern, the solution x„ is found to be given by 

= Xg + AnUg (1 + fl + fl H — a"" ) + r^ [s'q (1 + a + fl -i — a"") 
+ ^1 (1 + fl + fl^ + ■ ■ ■ fl""^) + ■ ■ ■ + £„_3 (1 + fl) + ^„_2 J ) 



1 " 

: Xq + VqAI h rgAt 

1 -fl 



1-fl 1-fl 1-fl 1-fl 



-0 1 ^''l 1 ^ ^^^,7-3 1 T^'^n-2 1 

1-fl 1-fl 1-fl 1-fl 



= Xg+TVg{l-a") + 

+ 7^Af [^o(l-fl"-') + ^,(l-fl"-') + --- + £-„_3(l-fl') + £-„_2(l-a)]. 

(6.68) 
The second line results from the sum of Eqs. (6.65). The third line applies the 
identity (6.10), and the last expression applies the relations 

At At r„At JoAt/t^ At r—— ^^ ^^^ 
= r, -^ — = T— 7^ — —^ = -\lDAt. (6.69) 



1-fl l-(l-A?/r) ' 1-fl l-(l-A?/z-) 

Means and Variances. The means of x„ and v„ follow from taking the means 
of Eqs. (6.66) and (6.68), respectively, 

(x„) = (x„) + r(uo)(l-fl"), (6.70a) 

(u„) = fl"(uo). (6.70b) 

The fluctuations of x„ and v„ are then found by taking the differences between 
Eqs. (6.68) and (6.70a), and (6.66) and (6.70b), 

x„ =Xa + TVoil-a") + 

r ^ 1 (6.71a) 

+ .^Z) A? [ £„ (1 - fl " ' ) + ^j (1 - fl " ^ ) + ■ • ■ + f„_3 {l-a-) + e^_2il-a)\, 



v„ = fl" V, + ^^ (fl""' £o+--- + a £„_2 + fl" £„_, ). (6.71b) 



We assume that x^ is independent of !;„ , and Xq and u,, are independent of all S/^. 
The variances of x„ and v„ can be obtained by multiplication and averaging of the 
corresponding fluctuations Xq and Vg . As shown in exercises 6.4. 1 and 6.4.2, we 
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obtain in this way the expressions 



■TUvA{\-a"Y+D 



nAt-2T(l-a") + T 



1 Zn 

I- a 
l + a 



x„v„) = T(v,')(l-a")a"+D 



I -a" 



l + a 



a iv„ + 



D l-a'" 
T l + a 



(6.72a) 

(6.72b) 
(6.72c) 



As shown in exercise 6.4.3, these relations contain the statistics of the diffusion 
model (6.63) as a specific case. 



6.4.3 Continuous Brownian Motion Statistics 



Continuous Statistics. A relevant question, which was already considered in 
Sect. 6.3. 1 regarding the position variance, is whether the Brownian motion model 
statistics represent a consistent continuous time model, or in other words whether 
n and the time interval At do only appear for A?/ r— > in the combination t = n At 
(the appearance of terms like n {AtY would indicate an inappropriate model for- 
mulation). First, we consider 



lim a" = lim I 1 = lim exp< « In 1 

Af^O Ar->0l ^ I A/->0 I f 



.expi-— ^ = e-"^ (6.73) 



Here, the logarithmic function was replaced by the first term of its Taylor series, 

At^ At 

T 



lim In 1 

A(^ I Z" 



(6.74) 



In terms of this limit for a" we obtain the means (6.70) in the limit A?/ r— > as 

(x) = (xo) + r(!;o)(l-e-"^), (6.75a) 

{v) = e-''{v,). (6.75b) 

Here, x„ and v„ are replaced by x and v, respectively, because we consider now 
functions of t. Therefore, these expressions represent indeed functions of t in the 
limit A?/ r — > 0. The consistency with the initial values can be seen by setting t = 
0. The means (6.75) are related in the continuous time limit by the relations 



d(x) 
dt 



(6.76) 
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The variances (6.72), too, become consistent functions of ^ in the limit A?/ r— > 0. 
In terms of Eq. (6.73) we find 



/x^ 



{XV 



■TUuA{l-e-"^f+DT 
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l-e- 



T 2 



l-e- 



-2tlT l~ 2 

e {v„ 



It 



(1-e ). 



(6.77a) 



(6.77b) 



{6.11c) 



We note that the term \ + a has to be replaced by 2. As done regarding the means, 
x„ and ;;„ are replaced here by x and u, respectively. The variances (6.77) obtained 
provide the correct initial values, as may be seen by setting t = 0. We note that we 
have the following relation in the continuous time limit. 



{xv\ 



\ d(x' 
2 dt 



(6.78) 



Hence, the covariance is controlled by the variance of x{t), such that there is no 
reason to consider the covariance regarding the following comparisons. 

Comparison with Diffusion Model: Asymptotic Limit. Let us compare the 
asymptotic features of the Brownian motion model and the diffusion model. In the 
limit A? / z" — > 00, the position statistics that are implied by the Brownian motion 
model are given by 



(x) = (X„)+T(Vn 



(x^ 



Dt. 



(6.79a) 



(6.79b) 



These results agree with the implications of the diffusion model (6.63), except that 
Eq. (6.63) applies a zero mean initial velocity because of x„ -x„_, = Z)"^ ^^„-i- 
For the velocity statistics the Brownian motion model provide for A?/ r— > oo 



D 

2t' 



(6.80a) 
(6.80b) 



Equation (6.80a) agrees with the consequences of the diffusion model. The availa- 
bility of Eq. (6.80b) represents a significant difference to the Brownian motion 
model, which does not offer such a relation (see the discussion on the properties 
of the Wiener process in Sect. 6.3.3). The advantage of Eq. (6.80b) is that we can 
use this so-called Einstein relation for the calculation of the diffusion coefficient 
D provided the characteristic relaxation time scale r and the variance are known. 
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For T, Einstein used Stokes' Law (see the discussion in Sect. 3.3.3) given by 
m 



67T lur 



(6.81) 



Here, r denotes the radius of a spherical particle, m is the particle mass, and // is 
the dynamic viscosity. The variance in Eq. (6.80b) is determined according to the 
Equipartition Law in one dimension, 

fr)4^. (6.82) 

Here, k represents Boltzmann's constant, and Prefers to the absolute temperature. 
The combination of the last two relations with Eq. (6.80b) does allow us then to 
calculate the diffusion coefficient, 

kT 



D- 



(6.83) 



3;r//r 

Therefore, the diffusion coefficient depends linearly on the temperature T if the 
models (6.81) and (6.82) are applied. 

Comparison with Diffusion Model: Transition to Asymptotic Limit. Next, 

let us compare the transition to the asymptotic limit according to the Brownian 
motion model and diffusion model. To apply the Brownian motion model we have 
to specify the initial velocity statistics <yo> and < u^ >. For simplicity, we assume 
that the initial velocity statistics are given by the equilibrium values 

D_ 

It' 



0, 



(6.84) 



Under these conditions, Eqs. (6.75b) and (6.77c) show that the mean and variance 
are also given by their equilibrium values, <u> = and <v^> = D I {2 f). Accord- 
ing to Eqs. (6.75a) and (6.77a), the mean and variances of x are then given by 



ir 



x„') + —il-e-"^y+DT 



T 2 



Xa-) + Dt + DT{l-e'"') 



l-e- 



■2 + 



l + e" 



(6.85a) 



(6.85b) 



x^\ + Dt-DT{\-e-'^'). 

To illustrate the behavior of the position variance, we introduce the normalized 
deviation from the initial variance 



B. 



Dt 



(6.86) 
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B„ 




Fig. 6.8. The solid line shows 5„. as function oftlr 
according to the Brownian motion formula (6.87). 
The dashed line shows B^f'^ according to Eq. (6.88) 
for the diffusion model. 



This quantity is given according to Eq. (6.85b) by 



T 



-1. 



(6.87) 



The corresponding behavior B^^^'^ according to the diffusion model, which is 
denoted by the superscript DM, is given by 



B,. 



(6.88) 



Figure 6.8 shows B^^ and -5^"*' ^s functions oftlr. The behavior of the Brownian 
motion model is the correct behavior, which is in agreement with measurements 
(Durbin 1983). The coarse diffusion model, which apphes large time steps At = r, 
does correctly describe the asymptotic behavior tl t^^ "x, but the prediction of the 
diffusion model is incorrect for relatively small 1 1 t, which is the most important 
period where most of the changes occur. In addition to the advantage described in 
the preceding paragraph, this comparison reveals another significant advantage of 
the Brownian motion model compared to the diffusion model. 



6.5 Population Dynamics 



Next, let us consider the stochastic modeling of the evolution of populations. In 
this way, we follow the organization of Chap. 5 by considering nonlinear models 
after the discussion of linear models. In particular, we will consider the extension 
of the logistic population model discussed in Chap. 5 to a stochastic population 
model. One purpose of the following discussion is to illustrate the scenarios that 
can be described if randomness is involved in nonlinear equations. Another pur- 
pose is the illustration of typical mathematical problems that appear in analyses of 
nonhnear stochastic equations. The analyses of many other problems lead to very 
similar questions, as given, for example, with regard to the application of non- 
linear stochastic equations for turbulence. 



6.5 Population Dynamics 233 



6.5.1 A Stochastic Logistic Model 

Logistic Model. The model considered in the following is the discrete logistic 
model (5.97), which we write in terms of P„ instead oiy„ used in Sect. 5.4.2, 

P^=P^_^+aP„_\l-^. (6.89) 

There are two equilibrium states P„ = and P„ = K that can be realized depending 
on the initial population PqI these equilibrium states imply P,^-P„_^ = 0, such that 
P„ will not change anymore. K is the carrying capacity, and the model parameter a 
determines the transition rate to the equilibrium state. To refer to the meaning of a 
we replace it by a = A? / T, where A^ denotes a time interval (which does not have 
to be small), and T is a characteristic time scale. By using the definition a = Ml T, 
we can write Eq. (6.89) as 

^"'^"-^ =^fl-^l (6.90) 

Stochastic Logistic Model. The evolution of populations is often affected by 
fluctuations that may be caused by variations of the available food, the impact of 
natural enemies, diseases, or weather conditions. Here, we have two possibilities: 
we can randomize K or T. The randomization of AT is questionable from a concep- 
tual point of view because K is seen to represent an upper limit for the population 
density that can be supported with food over a long term in a given area: see the 
discussions in Sects. 7.4 and 7.5. The randomization of K is also questionable 
regarding the fact that a random model for K may result in a negative population 
density K, which does not make sense. To simplify the following discussion we 
assume that K= \. The influence of AT can be covered by dividing the population 
equation (6.90) by K and introducing a new variable P„l K, so that the population 
equation becomes independent of K. The consequence of setting K= \ is that the 
population is now bounded, < /*„ < 1. To account for the effect of randomness on 
the population dynamics we randomize There. The consideration of negative and 
positive values of the growth time T represents an appropriate mean to reflect 
varying conditions for a population development. However, the direct randomi- 
zation of T does represent a very questionable approach because of the potential 
problem to divide by a zero time scale value T. Thus, we will assume that T ' is 
normally distributed, 

r-'=// + (7^^. (6.91) 

Ar 
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Fig. 6.9. Three reahzations of the stochastic population model (6.92), where t 
refers to time in years. The simulation conditions are described in Sect. 6.5.2. 

We apply Aff„_| = (At)"^ s„_i as before, and // and cr denote model parameters that 
will be specified below. The combination of the population model (6.90) with this 
model for T ' and K= \ with leads then to the model 



'■-'-'-pA^-pJ,*'^^''-' 



At 



At 



(6.92) 



The stochastic logistic model (6.92) does not represent the only stochastic version 
of the logistic model. Different other stochastic models can be designed by using 
certain birth and death rate models in equations for the population density PDF 
(see Allen 2003). The structure of Eq. (6.92) corresponds to the structure of the 
diffusion model (6.28) because the derivative of the variable considered is propor- 
tional to AW„_i I At. The suitability of such an assumption is illustrated in terms of 
exercise 6.3.2. 

Realizations. Three realizations of the stochastic population model (6.92) are 
shown in Fig. (6.9). The realizations considered illustrate an interesting property 
of the population model. Without randomizing T ', all trajectories with Q < Pq<\ 
would be finally attracted by the equilibrium state P„= 1 . However, by accounting 
for randomness, it is possible that the equilibrium state P„ = will be realized, too: 
see Fig. 6.9a. There is no overshooting of the two equilibrium values as long as a 
is not too large. 



6.5.2 One-Point Statistics and Correlations 



To understand the characteristic model features let us consider the development 
of one -point statistics (the population density PDF, mean, and variance) in time 
and the correlations that are implied by the stochastic model (6.92). 
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P P P 

Fig. 6.10. The development of the population density PDF/(p) in time t according 
to the stochastic population model (6.92). The PDF is shown at ^ = 0.1, ? = 0.5, and 
t=lm (a), (b), and (c), respectively. 



PDF. The only way to investigate the nonlinear stochastic model statistics is to 
calculate the model solution numerically. This was done by using the parameter 
values Pq = 0.25, // = 1, and o"= 2. The time step At = 0.01 was applied, and a 
number A^= 10* of realizations was used. The VDV f(p) was calculated at positions 
with a distance of 0.002 by using a filter size of 0.02. Here, /» refers to the sample 
space population density. The evolution of the population density PDF in time 
t = nKt is illustrated in Fig. 6.10. At ? = 0, there is a delta function peak at 
/» = 0.25. After a relatively short time t = 0.1, this delta peak is significantly 
distributed. The population probability flows then toward the equilibrium values 
/» = and/» = 1. Ki p = 0.5, there is a nonzero probability of population values 
over the entire domain. The population probability flows toward/) = 0, but there is 
no way to go beyond this value. This leads to the maximum of the population PDF 
j{p) close to/7 = 0. For later times, the continuing flow of the population probabil- 
ity toward /? = and p = \ leads to the development of sharp maximum values 
close to/7 = and/7 = 1. Asymptotically, the PDF/(/») develops delta-peak spikes 
close to/» = and/» = 1, and the probability for other population values vanishes. 

Moments. Next, let us analyze the mean and variance. In preparation of the 
derivation of an equation for the mean value of P,„ we write Eq. (6.92) as 



where t^W„_y = (A?)"" £■„_, is applied. By averaging this relation, we obtain 

(^, ) = (^,-1 ) + // A? {P„_, (1 - P„_, )) = (! + // M\P„_, )-^M {P„_ 



(6.93) 



(6.94) 



The term involving £■„_[ does not provide a contribution to this equation because 
£•„_! is independent of P„_i (P„_, does only involve terms like £„_2, £„-}, ...). There- 
fore, we have <P„_i (P„_, - 1) £„_i> = <P„_, (P„_i - 1)> < £„_i> = 0. Equation (6.94) 
for <P> is unclosed due to the appearance of the second-order moment <P„_i^>. 
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Fig. 6.11. The mean population density and standard deviation determined by the 
stochastic model (6.92) are shown in dependence on time t (in years). 



Let us derive an equation for the calculation of the second-order moment <P,; 
By squaring Eq. (6.93) we obtain 

^/ =^,->' +2/'„_r(l-P„_,)(//A? + a(A0"'^„_,) 



(6.95) 



After averaging this relation we find that the second-order moment of P„ is deter- 
mined by the equation 



P') = {pj) + 2^MIPJ{\-P„_,)) + ^'{M)HP,J{\-P„_,f 



+ a'AtlpJ{l-P,,J 



(6.96) 



This expression was obtained by neglecting terms that involve s„_i, but the term 
involving e„_^ provides the nonzero last contribution. Regarding the limit A^ — > 
we observe that the third term on the right-hand side can be neglected if A? — > 0. 
Equation (6.96) includes unknowns in terms of the third-order and fourth-order 
moments involved. Equations for the third-order and fourth-order moments do 
again involve higher-order moments. Hence, a closed system for the calculation of 
the mean and the variance of P„ cannot be obtained. The development of the mean 
population density and standard deviation in time t = n At is shown in Fig. 6.11, 
where A' = 10' realizations are applied. Both the mean and the standard deviation 
become constant asymptotically. The curve of the mean indicates that about 60% 
of the realizations do finally realize the value P„= 1. 

Correlations. The normalized correlation function of population densities at 
t = n At and s = mAt\s defined by 



C{t,s) = 



P{t)P{t + s) 

horn 



(6.97) 
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Fig. 6.12. The nonnalized correlation function (6.97) as a function of s ^ m At in 
years for several fixed values t = nAt (in years). 



Here, the value of t is considered to be fixed, this means C{t, s) is a function ofs. 
Figure 6.12 shows C(t, s) at the t values considered in Fig. 6.10 and other t values. 
The overall picture is that C(t, s) becomes constant after a transitional stage. The 
reason for this is the lack of a decorrelation mechanism as given by the relaxation 
term -v„_i / r in Eq. (6.59b). The limit C„(t) of C(t, s) for the case .s — > oo is shown 
in dependence on t in Fig. 6.13. It is interesting to see that C^(0 > 1 for very small 
values of t. This behavior is caused by the fact that the fluctuations are initially 
equal to zero (see Fig. 6.1 lb). The generation of fluctuations then implies relative- 
ly large correlation values. In the next stage we see a decrease of the C^{t) values 
until the minimum at ? = 1.25 is reached. This stage is characterized by a rapid 
distribution of the population PDF: see Fig. 6.10. This distribution of probability 
implies a correlation decrease (the same behavior is found for the evolution of a 
normal PDF with increasing variance). The reason that C„(?) does not continue to 
decay for increasing t values is given by the equilibrium values p = and p ^ I 
that confine the distribution of probability. Thus, the stage ^ > 1.25 is character- 
ized by an increase of C^(0 until the value one is reached. This is the stage where 
the peaks close to the equilibrium values p = and /? = 1 develop, this means the 
correlation of population values has to increase again due to the trend of realizing 
the equilibrium values. 
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Fig. 6.13. The hmit C^(t) of the standardized cor- 
relation function C(t, s) for the case i' — > oo. 



6. 5. 3 Model Application 



Time to Extinction. Let us calculate the time to extinction to illustrate the use 
of the stochastic population model. The time to extinction will be defined here to 
be the earliest time at which a realization is found in the interval <P„ <Z, where 
L = 10 **. The idea of using such a small L value is to perform this calculation in a 
close agreement with the exact theoretical result. The influence of variations of L 
on the time to extinction PDF is described in terms of exercise 6.5.5. 

Time to Extinction PDF. The time to extinction is a random variable, which is 
different for every realization. To characterize this variable, its PDF/(r) has been 
calculated. Here, r refers to the sample space time to extinction in years. The PDF 
J{t) was calculated at positions with a distance of 0.1 by using a filter size of 1, 
where A^= lO'' realizations were applied. The differences to the values used for the 
calculation of the population density PDF/(/7) arise from two facts. First, the range 
of data values is much larger for J{t). Second, there are less sample values 
available for the calculation of/(r) because only a part of all the realizations will 
approach the equilibrium value/? = 0. The time to extinction PDF/(z") is shown in 
Fig. 6.14. Here, the same model settings are used as for the population density 
PDF Jip), except that variations of a are considered. The behavior of the time to 
extinction PDF can be explained by having again a look at Fig. 6.10. There are 
two stages. First, the number of population density values that are relatively close 
to p^O steadily increases in time because the population probability flows toward 
the equilibrium point p = Q. The increasing number of zero-population values in 
time is reflected by an increase of the time to extinction PDF with the character- 
istic arrival time r at zero until the maximum is reached. The second stage shows 
a decrease of the time to extinction PDF with r because of the lack of new popu- 
lation density values that approach p = Q. The development in the second stage 
takes place at a lower pace because it is not driven by a probability flow like the 
development in the first stage. Therefore, the time to extinction PDFs are skewed. 
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Fig. 6.14. The time to extinction PDF/(z") implied by the population model (6.92). 
The values a^ 1.5, a^ 2, and a= 2.5 are applied in (a), (b), and (c), respectively. 

Figure 6.14 reveals that crhas a significant influence on the time to extinction 
PDF: the higher the noise intensity a, the smaller is the typical time to extinction. 
The value a^2 was chosen here as a reference value because the findings for this 
case are similar to the results of a population viability analysis for the endangered 
island fox (golden eagle predation is the primary cause of island fox mortality) 
(see Bakker at el. 2009). Why is the Fig. 6.14a PDF more noisy than the other 
curves? For this relatively low cr value we have only 6.9% of all realizations that 
reach the equilibrium point/) = 0. For 0"= 2 and 0"= 2.5, we have the case that 
39.8% and 54.1%) of the realizations reach the equilibrium point/? = 0. 

Initial Population Effect. It is interesting to consider the influence of the 
initial population density Pq on the time to extinction PDFy(z). This question is 
addressed in terms of Fig. 6.15 that shows the time to extinction PDF/(z) for 
different initial population values Pq. It may be seen that the influence of Pq onJ{T) 
is very limited. The PDF becomes more noisy with increasing Pq. The reason for 
this is given by the number of available realizations. For the cases considered, we 
have 39.8%o, 18.5%o and 5.9%o of all realizations that are involved in the time to 
extinction PDF calculation shown in (a), (b), and (c), respectively. 




10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 

T T T 

Fig. 6.15. The time to extinction PDFy(z) is shown for Pq = 0.25, Pq = 0.50, and 
Pq = 0.75 in (a), (b), and (c), respectively. All PDFs are obtained by using o"= 2. 
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Fig. 6.16. The dependence of the mean time 
to extinction r„, on the initial population Pq. 
The dots show the results obtained by using 
the population model (6.92). The line shows 
the approximation (6.98). 



To see the effect of the initial population density Pq more clearly, let us consider 
the effect of Pq on the mean time to extinction r,„. Instead of considering t„, as a 
function of Pq we look here at the equivalent function Po(t,„) because Po(t„,) varies 
from zero to one. The variation PqCO is presented in Fig. 6.16. The dependence 
Po(t,„) is plausible. The mean time to extinction t„, increases with Pq because it 
takes longer to realize the equilibrium value p = 0. For very small (very high) Pq, 
one has to expect that the t„, values become very small (very high). The analytical 
approximation 

0.99 



Po=0.01 + 



1 + e 



-2.6(r.,-14.04) 



(6.98) 



describes the variation ofPQ{T„,) very well. 



6.6 Summary 



The basic goal of this chapter was the explanation of how randomness should 
be involved in the deterministic difference equations considered in Chap. 5. For 
doing this we need knowledge about an appropriate structure of noise models and 
the influence of noise models on the statistical properties of a process considered. 
Finally, there is the question of how we can evaluate the suitability of a model for 
a stochastic process. Let us summarize the observations made in this chapter with 
regard to the latter questions. 

Noise Model. The definition of an appropriate noise model requires two sorts 
of information: we have to define the PDF of noise values, and we have to define 
the scaling of the noise process with time. We did only consider here noise that is 
normally distributed. According to the discussions in Chap. 4, this is the most 
natural assumption for many problems, but this approach may turn out to be 
inappropriate for the modeling of variables that are non-negative or bounded by 
zero and one. Regarding the scaling with time we concluded that the consideration 
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of AW„ = (A?)"^ £■„ is the appropriate choice for scaled noise variables, where £„ is 
normally distributed. The reason for that can be seen by considering the diffusion 
model x„ - x„_i = D"^ AfF„_,. This model provides a chance of x„ that scales with 
(A?)"^, which means that the variance of x„ changes by a contribution that scales 
with At. Hence, after n time steps we have a variance that is proportional to n At. 
This means that we calculate the variance correctly as a function of time t ^ n At. 
However, the use of AW„ = (A^)"^ £•„ for modeling the noise implies to deal with 
the problematic property of AW„ that 



A^„ AW,„ 
At At 



1 -f 

— II n = m 

At . (6.99) 

if n ^ in 



This relation means that the variance of AW„/ At goes to infinity as At — > 0. 

Effect of Noise on Model Statistics. A general observation of the discussion of 
processes in this chapter is that the normality of noise does result in normally 
distributed process variables, provided that the equation considered is linear and 
the initial values are normally distributed. This conclusion does also apply to the 
linear equation system (6.101). In particular, we have to ask whether the use of 
AW„ with the property (6.99) means that we develop models with poor statistical 
properties. The suitability of using AW„ as scaled noise variables was justified by 
considering the diffusion equation 

I — AW 
ylD—-^^. (6.100) 



At At 

It is correct that this model is a questionable model for the velocity (the left-hand 
side), but it was shown that the model represents a valid model for the particle 
position x„. A better model is given by the Brownian motion model (6.59) 
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(6.101a) 
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^^^^1^\. (6.101b) 

At rV ""' At J 

This model represents a sound model for both the velocity v„ and position x,„ but 
not for the particle acceleration (the left-hand side of Eq. (6.101b)). To have a 
good model for the particle position, velocity, and acceleration, one must consider 
a model where the derivative of the acceleration is driven by AW„_t I At (see 
Heinz 2003). Corresponding conclusions apply to the population model (6.92), 

^^^ = ^,-,(l-^,-.(/' + ^^) (6.102) 

which represents an appropriate model for the variable P„ considered. 
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Evaluation of Stochastic Models. The evaluation of the suitabihty of models 
for stochastic processes requires the evaluation of two sorts of model properties: 
the one-point statistics (PDF, mean, and variance), and correlations. The one-point 
statistics provide information about the typical value and range of variations of 
any variable considered. Correlations provide information about the characteristic 
lifetime of fluctuations, which is relevant to the modeling of processes. The evalu- 
ation of these properties is relatively simple as long as the difference equation 
considered is linear and the model parameters (the noise term and initial values) 
are normally distributed: we have then analytical expressions for the PDF and the 
correlations. The analysis of nonlinear equations is significantly more challenging. 
It is impossible to determine analytically the shape of the PDF, the equations for 
moments are unclosed due to the appearance of higher-order moments, and it is 
impossible to find an analytical expression for correlations. In the latter case, the 
statistical properties of processes considered have to be studied by simulations, as 
demonstrated with regard to population dynamics modeling in Sect. 6.5. 



6.7 Exercises 



6.2.1 Consider the difference equation y„^ ay „_t+b + r£„_i discussed in Sect. 6.2. 

a) Derive the difference equation for the mean <y>. Use this equation for 
the calculation of the equilibrium value of <>>„>. 

b) Derive the difference equation for the variance <y^>. Use this equation 
for the calculation of the equilibrium value of < j ^ >. 

c) Use the variance equation to explain why the equilibrium variance ofy,, 
is larger than the variance /'^ of the noise process r £/.. 

6.2.2 Consider again the equation y„ = ay„_i+h + r £„_i . The initial value is given 
by yo = So. Assume that this equation is incorrectly solved by using for each 
« = 1, 2, . . . always the same set of random numbers £„. 

a) Calculate analytically the mean value <y> for this case. 

b) Calculate analytically the standard deviation <y ^ >"^ for this case. 

c) Calculate analytically the correlation < y^y^^ > for this case. 

d) What is the difference between these results and the correct results for 
the case that < a < 1 and « — >■ oo? 

6.2.3 Consider the equation >>„ = 0.2 y„_i + 0.1 + £„_i, where jo is normally distrib- 
uted with zero mean and standard deviation 0. 1. The random variable £„_i is 
normality distributed and characterized by <£/> = and <£^£,> = 5^,„. 
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a) Graph three realizations of >>„. 

b) Graph the numerically calculated mean <y> for A' = lO'' realizations as a 
function of «. 

c) Graph in the same figure the corresponding theoretical result. 

d) Graph the numerically calculated standard deviation <y ^>"^ for A^ = 
10'' realizations as a function of «. 

e) Graph in the same figure the corresponding theoretical result. 

6.2.4 Consider again the stochastic model considered in exercise 6.2.3. 

a) Graph the numerically calculated filtered PDF at « = for A' = 10'' reali- 
zations. Use a filter interval of 0.05. 

b) Graph in the same figure the corresponding normal PDF at « = 0. 

c) Graph the numerically calculated filtered PDF at « = 1 for A' = 10'' reali- 
zations. Use a filter interval of 0.05. 

d) Graph in the same figure the corresponding normal PDF at « = 1 . 

e) Explain the reason for the difference between the PDF at « = and the 
PDFat«= 1. 

6.2.5 Consider again the stochastic model considered in exercise 6.2.3. 

a) Graph the numerically calculated normalized correlation function Co(«) 
= < y^y^^ > / <y^'>iorN^ 10'' realizations. 

b) Graph in the same figure the corresponding theoretical result. 

6.3.1 Apply Eq. (6.12) for y to show that the correlations of the random walk 
model (6.28) are characterized by < y,Xy„+,„ -y,,)-^^^- 

6.3.2 Consider the diffusion model x„ - x„_i ^ d At £•„_,. The random number £„_i 
is normality distributed and characterized by <£)> = and <£,, £,„> = S,,,,,, 
and d isa diffusion coefficient. 

a) Calculate the variance of x„. 

b) Present the variance obtained as a function of time t ^ n At. Explain the 
problem related to the use of this variance. 

6.3.3 Show that the function 

approaches for o"— > a delta function, /(y) — > S(y-yo). 

6.3.4 There is an instantaneous emission of a mass of 0.1 kg from a point source 
at a height of 10 m. The mean wind velocity in x direction is 10 m / s. The 
diffusion coefficient is given by the mean wind velocity in x direction mul- 
tiplied with the height of 10 m. 
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a) Calculate the x position at which the concentration has a maximum. 

b) Calculate the maximum concentration at this position. 

6.3.5 Consider the diffusion model y„ - j„_, = Z)"^ Aff„_„ which was discussed in 
Sect. 6.3. The old position is j„_, > 0. The new position y„ obtained by 
solving the diffusion equation is found to be beyond a totally reflecting 
boundary at j' = 0, this means y„<0. 

a) Calculate the mean velocity v over the time step At. 

b) Find the time ?i, which is the time required to reach the boundary. Find 
the time ?2, which is the remaining time over the time step At. 

c) Calculate the new position jVn+i, which is obtained after moving with the 
velocity v over the time ?2 in the positive y direction. 

6.3.6 Consider the diffusion model y„ - y„_i = D^'^ AW„_i from Sect. 6.3. Assume 
that there is a totally absorbing boundary. 

a) Show that the integral over the concentration (6.53) from zero to infinity 
is given by the following formula, which describes how much material 
is left in the accessible domain < j' < co, 



\C„dy = —j= j e ' ds. 

V^ _ 



b) Consider the data of the diffusion problem described in exercise 6.3.4. 
Calculate the total amount of mass in the domain 0<j<coatx= 10 m. 
Hint: use the error function (see Sect. 4.3.2) to calculate the integral. 

6.3.7 Consider the diffusion model j„ -;vi ^ D"^ AW„_i from Sect. 6.3. 

a) Derive an analytical formula for the concentration distribution that can 
be used for partially absorbing and partially reflecting boundaries. Show 
that this formula covers the cases of a totally reflecting boundary, of no 
boundary, and of a totally absorbing boundary. 

b) Use this formula to find the position of the maximum and the maximum 
ground concentration for a boundary with 50% reflection. 

6.4.1 Prove the following identities: 

a)(l-a ) +(l-fl ) H i-(l-fl) +(l-fl) = n-2- 



Y-a 1-a' 



b) a"-' (1 - a"-') + a"-^ (1 - a""') + ■ • ■ + a' (1 - a') + a (1 - a) 



l-a" 1- 



2n 



a 



l-a l-a' 
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6.4.2 Prove Eqs. (6.72) for the statistics of the Brownian motion model: 

l-a 



a) {x„') = {x;) + TUd„^](l-a"y+D 



b) (x„v„) = T{v,'){l-a")a'-+D 



-. /~2\ 2„/~2\ Dl-a^" 



1-fl" 



wA^-2r(l-fl") + r- 

1 + a 



1 + fl 



T 1 + fl 

Hint: use the identities given in exercise 6.4.1. 

6.4.3 Consider Eqs. (6.70) and (6.72) for means and variances of the Brownian 
motion model. For « = 0, these relations represent identities for the corre- 
sponding initial values. 

a) Specify these relations for « > 1 for the case that the Brownian motion 
model is reduced to the diffusion model (6.63). 

b) Calculate the initial mean and variance of the velocity by setting « = in 
the relations for « > 1 . This is a consistency condition if At — > 0. 

c) Specify the relations from a) by using the initial values obtained in b). 

6.4.4 Consider the Brownian motion model (6.59). 

a) Derive the discrete correlation function <£'„£'„+„,>, where « = 0, 1, ... 
andff! = 0, 1, .... 

b) Set t = n M and s = m At. Specify the correlation function obtained in a) 
to find the correlation function < v{t) v{t + s)>m the limit A? — > 0. 

c) Integrate the normalized correlation function <v{t)v{t + s) > I <v{t)'> 
over s from zero to infinity. 

d) What does this result mean regarding the relevance of r? 

6.4.5 Consider the Brownian motion model (6.59). 

a) Prove the identity 

-a )(l-flfl ) + (l-fl )(l-flfl )H h 

1 n 1 2n 

+ (1- fl")(l-fl"'fl") + (1 -fl)(l-fl'"fl) = «-(! + a'") h fl™ r-. 

1-fl 1-fl 

b) Apply Eq. (6.71a) and the identity given in a) to derive the discrete cor- 
relation function < ^„^„+,„ >, where « = 0, 1, ... and m = 0, 1, .... 

c) Set t = n At and s = m At. Specify the correlation function obtained in a) 
to find the correlation function < x(t) x{t + s)>i\\ the limit At — >• 0. 

d) Consider the correlation function < x(t) x{t + i') > for ? — > oo. 

e) Interpret the result obtained in d). 
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6.5.1 Consider Eqs. (6.94) and (6.96) for the first-order and the second-order 
moments of the population density of the stochastic population model. 

a) Specify these relations for the equilibrium case « — > co. 

b) Specify the relations obtained in a) for the case that A? — > 0. 

c) Use the relations obtained in b) to write the standard deviation as a func- 
tion of the mean. 

6.5.2 Consider the stochastic population model (6.92). Solve this equation nu- 
merically by following the settings described in Sect. 6.5.2. 

a) Graph three realizations that apply the same a. 

b) Graph three realizations that apply (7= 0.5, cr= 2, and a= 7.5. Use the 
same set of random numbers in all the three realizations. 

c) Comment on the effect of a. 

6.5.3 Continue with exercise 6.5.2. 

a) Graph the mean and standard deviation that are shown in Fig. 6.11. 

b) Calculate the so-called intensity of segregation, which is defined by 
S„-<P_^'>/[<P>il-<P>)l 

6.5.4 Consider Eqs. (6.94) and (6.96) for the first-order and the second-order 
moments of the population density of the stochastic population model. 

a) Use the beta PDF to parametrize third- and fourth-order moments in 
Eqs. (6.94) and (6.96) in terms of first- and second-order moments. 
Hint: use the moment relation in exercise 4.3.10 for doing this. 

b) Use these parametrizations in Eqs. (6.94) and (6.96) so that Eqs. (6.94) 
and (6.96) represent closed equations for first-order and second-order 
moments. Solve these equations by using a time step A? = 0.01. Graph 
the mean population and standard deviation that follow from the 
solution of these equations as function of «. 

c) Compare these curves with the exact mean and variance calculated in 
exercise 6.5.3. Discuss the suitability of using the beta PDF to close the 
moment equations. 

6.5.5 Consider the time to extinction PDF calculated from the stochastic popula- 
tion model in Sect. 6.5.3. 

a) Graph the time to extinction PDF given in Fig. 6.15a by following the 
settings used in Sect. 6.5.3 (use Pq = 0.25, // = 1, a= 2, At = 0.01, and 
L = 10-'). 

b) Graph the corresponding time to extinction PDFs for the cases Z = 10^ 
andi^lOl 

c) Explain why the influence of Z variations on the time to extinction PDF 
is reasonable. 
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Deterministic changes were discussed in Chap. 5 by analyzing several types of 
equations for changes (linear first-order equations, linear second-order equations, 
and nonlinear first-order equations). The goal of this discussion was to understand 
the nature of processes that are described by a given equation. In this chapter we 
ask how we can model any observed process in terms of equations. In particular, 
we are interested in an understanding of the laws that explain the development of 
any process. Is there a difference to the question considered in Chap. 5? A simple 
answer to this question is that an equation can describe several processes (see Fig. 
5.4 that shows processes provided by a second-order equation), while the model- 
ing of an observed process usually involves several equations (as given regarding 
the nonlinear and linear pendulum equations discussed in Chaps. 3 and 9). Hence, 
there is a difference to the approach used in Chap. 5. A more precise answer to 
this question is: The analysis of many processes enables the derivation of equa- 
tions from observations. Such equations may be new equations. It may be also the 
case that such equations are so complicated that their use requires systematic sim- 
plifications. Thus, the analysis of observed processes does often end up with new 
equations and a hierarchy of simpler equations that can be applied under specific 
conditions - which is different to the approach of Chap. 5 to work with one given 
equation. The approach applied to address the modeling of processes is the fol- 
lowing: First, we will focus on basic processes (those that have an increasing 
intensity, processes that level off, oscillating processes). Second, we only consider 
single variable problems here (the modeling of processes involving several varia- 
bles will be addressed in Chap. 9). Third, we consider only two sorts of processes: 
mechanical processes (for which we have well established laws) and population 
ecology processes (for which we have an ongoing debate about the existence of 
general laws). 
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Fig. 7.1. Basic scenarios of the development of 
a population P in time V. (a) continuous growth, 
(b) balance, (c) oscillation, and (d) collapse. 



This chapter is organized in the following way. Section 7.1 will describe basic 
scenarios of the development of processes and characteristic questions regarding 
the modeling of such processes. The modeling of mechanical processes with and 
without oscillations are addressed in Sects. 7.2, and 7.3, respectively. The model- 
ing of processes in population ecology with and without oscillations is the con- 
cern of Sects. 7.4 and 7.5, respectively. Section 7.6 will summarize the features of 
laws in mechanics and population ecology considered here. 



7.1 Motivation 



Population Dynamics. Processes that we have to consider usually reveal a 
variety of scenarios. An example for the wide range of process variations is given 
in Fig. 7.1 that shows possible developments of a population density (see the more 
detailed discussion in Sect. 7.5.1). Case 1 illustrates an unbounded population 
growth under optimal living conditions. Case 2 illustrates the development of a 
final equilibrium that is given by the maximum population that can be supported 
by the environment. Case 3 illustrates damped oscillations about an equilibrium 
state, which can be related to a delayed response of the supporting environment to 
the growing population (see the explanations in Sect. 7.5.1). Case 4 illustrates a 
breakdown of the environmental system that causes a collapse of the population 
(collapse means a dramatic reduction of past population values). The difference 
between these four cases is given by the interaction between the population and 
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the supporting system ranging from a zero interaction (case 1) to a strong cou- 
pling (case 4). Such a variety of interactions makes it rather unlikely that all these 
cases can be modeled on the basis of only one model: at least the cases that do and 
do not involve oscillations will require different models (as will be shown below). 
On the other hand, a helpful observation is that similar basic scenarios can be 
found for other processes. With regard to mechanical processes, for example, we 
often find processes that are characterized by the cases 2 and 3. Examples for such 
processes are given by the balance of temperature differences and motions of a 
spring-mass system, respectively. 

Questions Considered. The development of mathematical models for the cases 
illustrated in Fig. 7.1 (and for many other cases) leads to questions like: 

• How can we derive model equations? 

• Which cases can we cover on the basis of one equation? 

• Can we use the same equations for similar processes in different applications? 
From a more general point of view there are questions like: 

• What are the characteristic properties of a law? 

• Are there laws of mechanics? 

• Are there laws of population ecology? 

These and other questions will be addressed in the following based on the consid- 
eration of basic mechanical processes and population ecology processes 



7.2 Heat and Mass Transfer: Balance 

Let us discuss first a relatively simple but very important case: the modeling of 
the distribution of heat or mass. The basic problem is usually the modeling of the 
transition from any initial state (the emission of heat or mass from any source) to 
an equilibrium state (at which heat or mass distributions do not change anymore). 
Such processes may be well described by first-order linear differential equations. 



7.2.1 Heat and Mass Transfer Models 

Mass Transfer Model. How can we model the transfer of mass? Let us say j is 
the mass of a substance in a tank. We do not consider any spatial distribution of 
mass, but the mass is characterized by one global value. Then, the change ofy in 
time t can be characterized by the equation 

dt I dt ;,,. I dt L... 



250 7 Deterministic Evolution 



We consider a differential equation because the process considered is a continu- 
ous function of t. According to this relation, y changes due to two contributions: 
the inflow rate {dy I dt)j„ which increases y, and the outflow rate {dy I dt)^,,„ which 
decreases y. To find an equation for y we have to relate the inflow and outflow 
rates to y. Regarding the outflow rate, it is a reasonable assumption to assume that 
(dy/dt)g„f is proportional to y, which means that 

r is a time scale that characterizes the outflow. The need to involve a time scale in 
this relation arises from the requirement to design a dimensionally correct equa- 
tion. It is also reasonable to consider a similar structure for {dy/ dt)j,„ 

Here, y^, is a variable that has the same dimension as y. The combination of Eqs. 
(7.1), (7.2), and (7.3) then leads to the equation 

± = -1^1^. (7.4) 

dt T 

This equation explains the physical relevance of y^: y^ is an equilibrium solution 
because there is no change ofy anymore ify = y^. The model parameters j^, and T 
have to be provided to have a model for y. Both y^ and T are assumed to be inde- 
pendent ofy, but these parameters can be functions of t. 

Heat Transfer Mechanisms. Let us consider heat transfer to understand which 
sort of equation is needed to describe this process. It is important to know that 
there is not just one heat transfer mechanism, but there are several processes that 
may take place. A first mechanism is convection that takes place through diffusion 
(the random Brownian motion of individual particles in a fluid) and advection (the 
transport of heat by the larger-scale motions in the fluid). This means, convection 
is the transport of heat by the actual movement of warmed matter (for example, 
the movement of heat that leaves a hot cup of coffee as the currents of steam and 
air rise). A second mechanism is conduction, which is the transfer and distribution 
of heat energy from atom to atom within a substance. For example, a spoon in a 
cup of hot soup becomes warmer because the heat from the soup is conducted 
along the spoon. A third mechanism is thermal radiation, which is the transfer of 
heat energy through empty space due to electromagnetic waves. An example is 
given by sunlight that is radiated through space to the Earth. In the following we 
will only consider the mathematical modeling of convection. For doing this we 
consider the heat transfer in still air where the temperature difference is relatively 
small (about 20°K - 30°K). 
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Heat Transfer Model. To explain the assumptions that underlie Newton's Law 
of Cooling let us consider the following equation for the transfer of temperature j, 

^ + C/^ = 0. (7.5) 

dt dx 

This equation represents a partial differential equation (readers who are unfamiliar 
with such equations can skip the following explanation of how Eq. (7.7) can be 
derived). Equation (7.5) is the result of simplifying a more general temperature 
equation that is derived in Sect. 10.5.2. The underlying idea of Eq. (7.5) is the 
assumption that a change of y in time is possible through convective transport 
along an axis x with a characteristic velocity U. Next, we assume that the spatial 
gradient dy I dx can be approximated by (y - y^ I Ax. Here, y^ is a characteristic 
external (ambient) temperature, and Ax characterizes the spatial scale of the heat 
difference. With this assumption, Eq. (7.5) reads 

f^ = _C;ZzZ.. (7.6) 

dt Ax 

In this equation, the partial derivative dy I dt was replaced by the total derivative 
dyldt because the derivative by t is the only derivative that is left here. By intro- 
ducing now the characteristic time scale T= Ax/ Uwe find 

!t=.y^y^, (7.7) 

dt T 

This equation represents Newton's Law of Cooling. It assumes that the change of 
y in time is controlled by the difference between y and the external temperature y^,. 
The validity of the negative sign can be explained in the following way: For the 
case J >y^, we find that jv decreases until jv becomes equal to j^^. Vice versa, for the 
case y<ye,y will increase until y becomes equal to y^. Equation (7.7) has the same 
structure as Eq. (7.4) for the conservation of mass. We assume again thatj;^ and T 
are independent of j', but they can be functions of t. 



7.2.2 First-Order Linear Differential Equations 

Next, let us consider the solution of Eq. (7.7). This question will be only con- 
sidered for the relevant case that the model parameters y^ and T are constant 
(techniques that allow the solution of Eq. (7.7) for the case that 3/^ is a function oft 
are described, for example, by Boyce & DiPrima 2009). The analytical solution of 
Eq. (7.7) was already obtained in Sect. 5.5.2 based on the corresponding differ- 
ence equation. The solution of Eq. (7.7) as a differential equation will be shown in 
the following to illustrate the approach for solving differential equations. 
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Solution. Equation (7.7) can be solved by means of the method of separation of 
variables. For applying this method we write Eq. (7.7) such that the variables y 
and t are separated, 

dy dt 



y-Ve T 

We can integrate this relation formally, 



(7.8) 



\^^ = Yi-C, (7.9) 

where C is any unknown constant. Next, we perform the integration on both sides, 

\n\y-yA=-j+C. (7.10) 

By taking both sides as exponents of an exponential fiinction we obtain 

\y-yA=e'e-'^'. (7.11) 

This relation can be also written 

y-y, =±e e =ce , (7.12) 

where the constant ±e'^ was replaced by another unknown constant c to simplify 
the notation. The constant c be determined by specifying Eq. (7. 12) for ? = 0, 

y^-ye^c. (7.13) 

Here, j'o refers to the initial value of j at ? = 0. By using this expression for c in 
Eq. (7.12) we find 

y = y. + e-'^'{y,-y^ (7.14) 

This expression recovers the result (5.120), which was obtained from the corres- 
ponding difference equation. 

Solution Check. It is always helpful to check the validity of solutions obtained 
for differential equations. This requires, first, evidence that Eq. (7.14) provides the 
correct initial value, and, second, evidence that Eq. (7.14) indeed satisfies the 
differential equation (7.7). The correctness of the initial value provided by Eq. 
(7.14) can be seen by setting t = Qm Eq. (7. 14), 

y{t = (i) = y,. (7.15) 

To see that Eq. (7. 14) indeed satisfies Eq. (7.7) we calculate the derivative oiy{t). 
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The last expression is obtained by replacing the exponential function according to 
Eq. (7.14). Hence, Eq. (7.14) does indeed solve the differential equation (7.7). 



7.2.3 Time of Death 

Problem. Let us consider an example to illustrate the use of Eq. (7.14) for the 
calculation of heat transfer. A police report provides the following facts: 
Police arrived at the scene of a murder at 8 a.m. They immediately took and recor- 
ded the temperature of the corpse, which was 33°C, and thoroughly inspected the 
area. By the time they finished the inspection, it was 10 a.m. They again took the 
temperature of the corpse, which had dropped to 29°C, and had the corpse sent to 
the morgue. The temperature at the crime scene had remained steady at 23°C. We 
suppose that the corpse temperature obeys Newton's Law of Cooling. What is the 
time of death by assuming a normal body temperature of 37°C at this time? 

Solution. To address this problem, we consider Eq. (7. 14) such that we account 
for an initial time tf^, which we try to calculate, 

j = j,+e-"-'°"^(j„-jJ. (7.17) 

The setting ? = ?o shows that y{t^ = y^. By taking the derivative of >> we can prove 
that Eq. (7.17) is also a solution of the differential equation (7.7), 

^ = -}^--'-(.o-.J-^, (7.18) 

at 1 1 

where Eq. (7.17) was used to obtain the last expression. The initial temperature 
j^o = 37°C, and the external temperature is j^ = 23°C. Hence, Eq. (7.17) reads 

>' = 23 + 14e"<'"''''"'. (7.19) 

We know that the temperature was 33°C at 8 a.m., and 29°C at 10 a.m., this means 
we have the relations 

33 = 23 + 14e"<'"'°'"', 29 = 23 + 14e""""'''"''. (7.20) 

The latter two conditions can be also written 

-(8-ro) = rhif^^l -(iO-r„) = nnf^^\ (7.21) 

By equating both implied expressions for t^ we obtain 

8 + rki(5/7) = 10 + nn(3/7). (7.22) 
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Hence, T is given by 

r = ? = ^ , = 3.92. (7.23) 

ln(5/7)-ln(3/7) In(5/3) 

We can use this value in Eq. (7.21) to find 

?o=8 + 3.921n(5/7) = 6.68. (7.24) 

Hence, the time of death was 6:41 a.m. The use of the values obtained for Tand t^ 
in Eq. (7.19) provides the temperature function 

J = 23 + 14e-<'-"*""^ (7.25) 

This formula is correct because it provides the correct temperature values 37°C at 
?o = 6.68, 33°C at 8 a.m., and 29°C at 10 a.m. 



7.2.4 Contamination of Lakes 

Problem. Let us consider a second example to illustrate the application of Eq. 
(7.14) to the calculation of mass transfer (Rainey 1967, Boyce & DiPrima 2009). 
A lake has a constant volume V (measured in km^) containing at time t a mass Q{t) 
(in kg) of pollutant. The initial mass at ? = is denoted by Q^. The pollutant is 
evenly distributed throughout the lake with a concentration c{t) = Q(t) I V. We 
assume that water containing a constant concentration c,„ of pollutant enters the 
lake at a rate r (in kmVyear), and that water leaves the lake at the same rate. Let us 
consider the following questions: 

(1) What is the mass Q{t) at any time f! 

(2) If the addition of pollutants to the lake is terminated (c,„ = 0), which time 
interval rmust elapse such that the mass Q of pollutants is reduced io Qf^ I al 

(3) Regarding the data given in Table 7.1, what is the time r necessary to reduce 
the contamination of each of these lakes to 10% of the original value? 

Solution. To find the answers to the latter questions we have to derive first the 
differential equation for Q{t). The dimensions involved imply that the change of 
Q{t) in time t is determined by the flow rate r multiplied by a concentration c. In 
particular, we have a balance between the inflow and outflow, 

-^ = ('"C) inflow - ('"C) outflow (7.26) 

The rate r and inflow concentration c„, are constants. The outflow concentration is 
given by the actual pollutant concentration c{t) = Q{t)l V. Hence, Eq. (7.26) reads 

^ = rc,„ - r^ = --{Q - VcJ. (7.27) 

dt V V 
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Table 7.1 Volume and flow data for the Great Lakes (Rainey 1967). 



Lake 


F(km') 


r (kmVyear) 


r (years) 


Superior 


12,200 


65.2 


430.9 


Michigan 


4,900 


158.0 


71.4 


Erie 


460 


175.0 


6.1 


Ontario 


1,600 


209.0 


17.6 



a 



This equation corresponds to Eq. (7.7): the variables Q, V c„„ and VI r used here 
correspond to the variables y, y^, and Tused in Eq. (7.7), respectively. According 
to Eq. (7.14) the solution of Eq. (7.27) is then given by 

Q = Vc,„+e-'"'{Q,-Vc,„), (7.28) 

which provides the answer to question (1). To find the answer to question (2) we 
apply c,„ = and use the condition 

^'-Q,e-'-'\ (7.29) 

The latter condition implies 

ln(l/«) = -y. (7.30) 

Hence, the answer to question (2) is given by 

r = — ki«. (7.31) 

r 

The answer to question (3) is given in Table 7. 1, where « = 10 is used. 



7,3 Newton's Laws of Motion: Oscillations 

Next, let us consider how another basic process can be modeled: damped or 
undamped oscillations between two equilibrium states. The basis for the modeling 
of this process is given by Newton's Laws of Motion. Although the focus in this 
chapter is on the discussion of modeling principles for single variable processes, 
Newton's Laws of Motion will be presented here for several variables in order to 
prepare the application of Newton's Laws to several variable cases in Chap. 9. 
Due to the use of vector notation, this approach does not add technical difficulties. 
The resulting equations will be used in this chapter only for single-variable cases. 
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= m" 




Fig. 7.2. An illustration of Newton's Third 
Law. The cannon has a mass M and accel- 
eration A, and the cannon ball has a mass 
m and acceleration a. The forces F^b and 
F(- of the cannon ball and cannon have the 
same magnitudes but opposite directions. 



7.3.1 Newton 's Laws of Motion 

Newton's Laws of Motion. Sir Isaac Newton developed in his famous book 
"Philosophise Naturalis Principia Mathematica" (1687), Latin for "Mathematical 
Principles of Natural Philosophy", a theory for the forces acting on a macroscopic 
body and the motion of that body. Newton's theory explains the motion of many 
physical objects and systems. For example, Newton demonstrated that these Laws 
of Motion, combined with his Law of Universal Gravitation, explain Kepler's Laws 
of Planetary Motion. Newton's Laws were verified by experiment and observation, 
and they are excellent approximations at the scales and speeds of everyday life. 
Newton's Laws of Motion, together with his Law of Universal Gravitation and the 
related developments of calculus, provided for the first time a unified quantitative 
explanation for a wide range of physical phenomena. Thus, Newton is seen by 
many scholars to be one of the most influential people in human history. Newton's 
Laws of Motion can be summarized by the following three laws: 

• First Law (the law of inertia): An object will remain at rest or in uniform motion 
in a straight line if it is not affected by an external force. 

• Second Law (the "force equals mass times acceleration" law): A body that is 
affected by a force F experiences an acceleration a that is related to the force by 
F = m a, where m is the mass of the body. 

• Third Law (the action-reaction law): For every action, there is an equal and 
opposite reaction. 

The bold symbols F and a refer to three-dimensional vectors, i.e., F = (T^i, Fj, F^ 
and a = {a^, 02, a^)- 

Newton's Third Law. Newton's Third Law means that all forces occur in equal 
but oppositely directed pairs: whenever a first body exerts a force F on a second 
body, the second body exerts a force -F on the first body. F and -F are equal in 
magnitude and opposite in direction. Newton used the Third Law to derive the Law 
of Conservation of Momentum (actually, conservation of momentum is the more 
fundamental idea). Newton's Third Law can be illustrated by considering a cannon 
that fires a cannon ball (see Fig. 7.2). The force Fi^g that acts on the cannon ball is 
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given by F^b = m a, where a is the cannon ball acceleration. The force F^ that acts 
on the cannon is given by Fc^ MA, where M is the mass of the cannon and A is 
the cannon acceleration. The two forces have the same magnitudes but opposite 
directions. Why does the cannon ball shoot out so far, whereas the cannon itself is 
only kicked back a httle bit? Because M» m, which implies that \a\ » \A\ due to 
m\a\^ M \A\. Another example for Newton's Third Law is given by the flight of a 
balloon: if air is rushing out of a balloon, the reaction is that the balloon is forced 
away. Newton's Third Law is also the reason why aircraft can fly (see Chap. 3). 
The air is deflected downward by the action of the airfoil, and in reaction the wing 
is pushed upward by the hft force. 

Newton's Second Law. Newton's Second Law provides an explanation for the 
relation between a force acting on a body and the motion of that body. In terms of 
the definition a = dv/dt of acceleration, Newton's Second Law can also be written 

4^ = -' (^-32) 

dt m 

or, by using the definition i; = dxl dt of velocity, Newton's Second Law reads 

^ = ^. (7.33) 

dt m 

Thus, for a given force F = F(dx/ dt, x, t), Newton's Second Law provides a dif- 
ferential equation of second order for the calculation of the position jc(0 of a body. 
Obviously, the velocity v = dx I dt can be calculated if x{t) is known. Equation 
(7.33) provides the basis for solving many everyday life problems. However, this 
equation cannot be applied if the mass m is changing, or if the object considered is 
traveling with a velocity close to the speed of light. Equation (7.33) does also not 
apply on the very small scale of atoms where quantum mechanics must be used. 

Newton's First Law. Newton's First Law is a consequence of Newton's Sec- 
ond Law for the case that F=0. Equation (7.32) implies for this case 

^ = 0, (7.34) 

dt 

which means that the velocity vector has to be constant, 

V - constant. (7.35) 

This writing means that both the magnitude and direction of the velocity vector 
are constant. Therefore, an object will remain at rest or in uniform motion in a 
straight line. 

Spring-Mass System. Vibrations represent a basic feature of many processes. 
There are mechanical vibrations (pendulum), electrical vibrations (the electrical 
current in a simple series circuit), electro-mechanical vibrations (microphone), 
optical vibrations (laser), biological vibrations (Lotka-Volterra equation), climatic 
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U+L 



F=-kL 
F„=me 



L„+L+y 



Fig. 7.3. An illustration of a spring-mass system. 

oscillations (El Nifio-Southern Oscillation), and chemical oscillations (Belousov- 
Zhabotinsky reaction). Let us consider a spring-mass system as a simple example 
for such oscillations and the use of Newton's Second Law for the calculation of the 
motion of bodies. The vertical spring-mass system is illustrated in Fig. 7.3. The 
original length of the spring without mass is Lq. The addition of a mass m causes 
an elongation L of the spring in the downward direction, which is considered to be 
the positive direction. Two forces act at the point where the mass is attached to the 
spring: the gravity force F^^ mg,g refers to the gravity acceleration, and a spring 
force F, acting to restore the spring to its natural position. The spring force can be 
described by Hooke's Law if the elongation L is relatively small. For this case, the 
spring force is proportional to the elongation, F^ = - k L. Here, A: is a positive 
constant, which is called the spring constant. A negative sign appears mF^ = -kL 
because the spring force acts in the negative upward direction. The mass is in an 
equilibrium, which means that the mass does not move. The condition for this 
equilibrium is that the two forces balance each other, which means that they add to 
zero. 



m 2 — k L = 0. 



(7.36) 



To study the motion of the spring system we use Newton's Second Law (7.33), 
F 



d'y 



(7.37) 



Here, y{t) is the displacement of the mass from its equilibrium position at time t. 
According to our assumption above on L^ and L, y is positive in the downward 
direction. F{i) represents the sum of all forces that act on the mass. In particular, 
F{t) is assumed to be given by 



■■F^+F,+F,. 



(7.38) 
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The three forces are related to dyl dt and j by the expressions 

F^=mg, (7.39a) 

F^=-k{L + y), (7.39b) 

F,=-r^. (7.39c) 

at 

The gravity force 7*'^ is unaffected by the displacement y of the mass. The expres- 
sion for the spring force 7*', is extended here by accounting for the varying distance 
y in addition to the equilibrium elongation L considered above, i.e., the spring 
force is assumed to be proportional to the total elongation L + y. As as-sumed 
above, we consider the elongation L + j to be sufficiently small such that Hooke's 
Law can be applied. In addition to F^ and F, we do also consider here a damping 
force Fj. This force acts opposite to the direction of motion of the mass: this force 
reduces the velocity dy/ dt. Here, ;Kis a positive constant that is called the damping 
constant. The structure of the damping force corresponds to Stokes' Law (see 
Sect. 3.3.3). The damping force may arise from the resistance from the air in 
which the mass moves. Interestingly, the assumption of a damping force that is 
linear in the velocity dy I dt is only one reasonable assumption among several 
possible choices. In general, there can be also other forces that contribute to the 
total force F{t), for example, any external force that implies forced vibrations. The 
combination of Newton's Second Law (7.33) with these assumptions about the 
forces involved then results in the following equation for the displacement j(?). 



d^y if ,,^ , dy 

— f = — mg - k(L + y)-r^ 
dt m dt 



, - ,mg-k{L + y)-y^\. (7.40) 



We multiply this equation with m and use the equilibrium condition mg = kL, 

m^ + r— + ky = i). (7.41) 

dt dt 

This equation represents a linear homogeneous differential equation of second 
order for the calculation of the displacement y{t). From a more general point of 
view, Eq. (7.41) represents the equation for a damped harmonic oscillator. Such 
oscillator equations occur in a diverse range of disciphnes, for example, in control 
engineering, mechanical engineering and electrical engineering. 

Undamped Harmonic Oscillator Solution. How can we solve the damped 
harmonic oscillator equation (7.41)? It turns out that there is no simple answer to 
this question. To see the structure of solutions, let us consider a relatively simple 
case given by the undamped harmonic oscillator equation 

^^-y = 0. (7.42) 

dt m 
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Due to dimensional reasons, the constant mik represents a squared time. Thus, we 
introduce the characteristic time scale T= (nilkf^, such that (7.42) can be written 

d'y y 



2+^ = 0- (7.43) 



A solution to this equation is given by any function y{t) having a second-order 
derivative ^"(O that is proportional to y{t). Exponential functions are characterized 
by the property y"{t) ~ y(t). A closer look at Eq. (7.43) reveals that one possible 
solution is given by 

yi{t) = e"". (7.44a) 

Here, i is the imaginary unit defined by /^ = -1 . Another possible solution is 

y,{t) = e-^'". (7.44b) 

The validity of these two solutions can be proven by calculating the second-order 
derivatives of ji and j2- A further consideration of this question reveals that any 
linear combination 

y,{t) = c,e"'^ +c^e-'"'' (7.44c) 

is also a solution to Eq. (7.43), where c, and C2 are any constants (the rewriting of 
the solution (7.44c) in terms of real-valued functions will be shown below). The 
observations made here lead to two relevant conclusions: First, exponential func- 
tions may provide solutions for a linear, homogeneous, second-order differential 
equation with constant coefficients. Second, the general solution can be a super- 
position of two exponential solutions. Consequently, the calculation of solutions 
for the damped harmonic oscillator equation (7.41) is not very simple: it requires a 
detailed analysis that will be presented in Sect. 7.3.2. 



7.3.2 Second-Order Linear Differential Equations 

Differential Equation. To find the solution of the damped harmonic oscillator 
equation (7.41) let us consider a linear, homogeneous, second-order differential 
equation with constant coefficients a, b, and c, 

a^ + b^ + cy = 0. (7.45) 

dt' dt ^ 

This equation corresponds to the oscillator equation (7.41): the difference is that 
the positive constants m, j, and k in Eq. (7.41) are replaced here by any (positive 
or negative) constants a, b, and c, respectively. The solution of Eq. (7.45) depends 
on two constants that are produced by the two integrations required to solve this 
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equation. To determine these two constants, we consider conditions for the initial 
value ofy and the initial value oidy/ dt, 

7(0) = Jo, (7.46a) 

y\Q) = y\. (7.46b) 

Here, jo and j'o are considered to be known parameters. 

General Solution. To find conditions for the existence of exponential solutions 
we assume a solution of the form 

y{t) = e'\ (7.47) 

where r is any unknown constant. By using this assumption in Eq. (7.45) we find 

{ar +br + c)e' =Q. (7.48) 

This condition is satisfied for all t if the parameter r satisfies the condition 

ar^ +br + c = Q. (7.49) 

The latter equation is called the characteristic equation of the differential equation 
(7.45). The two roots ri and Tj of the characteristic equation, which are called the 
eigenvalues, are given by 



J_ 
2a 



■\Jb' - Aac , r, = — -b- yjb' - 4ac 

J ' 2aL 



(7.50) 



Another appropriate way to write I'l and rj, which will be used for the presentation 
of solutions below, is given by 

i'i=''s+''d, r^=rs-''D, (7-51) 

where r^ and r^ are given by 



b 



2a 2 2a 



V^'-4ac. (7.52) 



The general solution of Eq. (7.45) represents a linear superposition of the two 
possible solutions exp(r, t) and exp(r2 1), 

y{t) = c,e'' +c^e-'. (7.53) 

The two unknown constants Ci and C2 can be calculated by the constraint that y{t) 
has to satisfy the initial conditions j(0) = y^ and dy/ dt(0) ^ y'o- The differentiation 
of Eq. (7.53) provides for the derivative ofy 

y\t) = Cir^e'''' +C2r^e~' . (7.54) 

The consideration of the last two formulas at ? = then provides the conditions 

Jq = C[ +C2. (7.55a) 

y\ = c^r^+c^r^. (7.55b) 
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We multiply Eq. (7.55a), first, with -rj and, second, with -r, and take the sum of 
both equations. This results in the relations 

c, = yj^Zhl^. (7.56a) 

c, = -ZJlZ^iZL. (7.56b) 

'1 - h 

With these expressions, the general solution (7.53) can be written 

3.(,) = Z:oZ^iZL/.'_ZWlZcL,'V. (7.57) 

n - '•2 ''1 - 'Iz 

The validity of this solution can be proven by showing that this solution satisfies 
both the initial conditions (7.46) and the differential equation (7.45). This result 
agrees with the conclusion (5.140) obtained by the solution of the corresponding 
difference equation. 

Solution Features. The features of this solution will depend essentially on Vi 
and r,. According to Eq. (7.50), we have to consider the three cases that we have 
two real eigenvalues (Z;^ - 4 a c > 0), one real eigenvalue (Z>^ - 4 a c = 0), and two 
imaginary eigenvalues {h^ - Aac < 0). Let us have a closer look at these cases to 
show the existence of real-valued solutions (the approach is very similar to the 
discussion of the corresponding discrete equation in Sect. 5.3). The first case is 
given for b^ - A a c > Q. The solution of the differential equation (7.45) is then 
given by Eq. (7.57), which can be written 

rxl i-^r I'll r^r 

y(t) = '-^^-y\- ''' "'''" yo- (7.58) 

n - '•2 1 - h 

Another convenient writing of the latter formula, which will be used for the dis- 
cussion of the second and third cases below, applies the relations r^= r^ + r^ and 
Tj = r^ - r^, for the eigenvalues r, and Tj, 

I'D I -I'D I 'S' -I'd' 

yit) = e' y\-^ -^ e' y,. (7.59) 

2'fl 2/'^ 

The functions exp(r£, t) and exp(-rfl t) can be rewritten in terms of the definitions 
of hyperbolic sine and cosine functions (Abramowitz & Stegun 1984), 

e" = cosh(r^ t) + sinh(rQ t), e '° = cosh(r^ t) - sinh(r^ t). (7.60) 

By using these expressions we can write the solution y{t) as 

sinh(r^ t) , (r^ - r, ) cosh(ro + (r^ + r, ) sinh(r^ t) , 
yit) = e Jo e yo. (7.61) 

'd 2r^ 
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The use of rj - rj = 2 r^, and ri + rj = 2 r^ enables the following simplification of 
this equation, 



sinhCr^O , 



sinh(r^O 
cosh(r^O r^ 



e Jo- 



(7.62) 



The second case is given for b^ - 4 ac ^ 0, which means that r^ = 0. The limits of 
hyperbolic sine and cosine functions for the case r^, — > are given by the relations 
(Abramowitz & Stegun 1984) 



sinh(r^O 
lim = t, 



lim cosh(;-o ?) = 1 . 



(7.63) 



Hence, Eq. (7.62) reads for this second case 



(7.64) 



The third case is given for Z^^ - 4 a c < 0, which means that r^, is imaginary. To 
deal with this case we write 



I'n =""„.. 



Here, r^,, is a real number given by 

r^, = — ^|-h +4ac. 
2a 



(7.65) 



(7.66) 



The hyperbolic sine and cosine functions involved in Eq. (7.62) can be written for 
this case (Abramowitz & Stegun 1984) 



sinh(/r£,.?) sin(r£„?) 



cosh(; r^,0 = cos(r^,0- 



(7.67) 



Hence, Eq. (7.62) reads for this third case 



sin(/vO ,;, , 



sin(/vO 
cos(r^. t) 1\ 



f^n* 



e Jo- 



y{t) = e"\ 



sinh(r^O 

y\+ 



sinh(r^O 
cosh(r^O r^ 



(7.68) 



The formulas obtained for the three cases can be summarized by 

J'o '/ ^^ ~ 4ac > 



ty\+(\-rst)yo 



sin(ra,0 

y\+ 



I'll* 



sin(r^.O 
cos(/-o, t) 1\ 



I'n, 



ifh-4ac = 0. (7.69) 
yo ifb'-4ac<0 



Hence, we obtain real-valued solutions for the three possible cases. 
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7.3.3 The Damped Harmonic Oscillator 



Damped Harmonic Oscillator Solution. A specific case of the general linear 
second-order equation (7.45) is given by the damped harmonic oscillator equation 



m 



d y dy 



, +Y^^ + ky = Q. 
df dt 



{I.IQ) 



The solution for this equation can be obtained by setting a = m,b = y, and c = ki\\ 
Eq. (7.69), 



y = e- 



Jst 



sinKr^O 

y\+ 



sinh(r^O 
cosh(roO r^ 



ty\+i^-rst)yo 



sin(ro.O , 

^0 + 



sin(r^,0 
cos(r^, t) }\ 



I'm 



Jo '/ r' - ^mk > 

ifr^-4mk = 0. (7.71) 
j^o if r^ - 4mk < 



According to Eqs. (7.52) and (7.66), the parameters r^, r^, and r^, are given by 

(7.72) 



r 

2m 



2m J m 



^n* 



I m I 2m 



Oscillatory Solutions. Let us discuss the solutions of the damped harmonic 
oscillator equation for the case of oscillations, i.e., for y^ - A mk <Q. According 
to Eq. (7.71), the solution for this case reads 



y- = e"\ cos(r^.O + >'« ^^^° sin(r^.O \- 



yo [ r„,y^ 

To simplify the discussion of this case, it is helpful to introduce an angle Shy 

y\-rsyQ 



(7.73) 



tan 5 = 



'fl.Jo 



For given Jo and j'o values, the angle Sis determined by the expression 



5 - arctan 



'•d-Jo 



(7.74) 



(7.75) 



The advantage of introducing S is given by the possibility to rewrite the solution 
(7.73) in the following way (Abramowitz & Stegun 1984) 



yo 



j'^' (cos(r^,0 + tan(?sin(r^.?)j 



,j , cos(r^, t)cosS + sin(r^, t) sin S ,.^ , cos(rQ, t - 5) 



(7.76) 



cos (J 



COS J 
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Nondimensional Oscillatory Solutions. To see the damping effect on j in a 
better way we introduce the frequency cOq = {k/ inY'^ of undamped motion and the 
damping frequency (0^= y I {2m). In addition, we introduce the dimensionless time 
t,^C0(ft and the dimensionless ratio r^ = o^/ Oo- By adopting these variables, r^ t 
and Tr,. t can be written 



r^t 1 - ■ 

2m 



CO, 



CO. 



-rj.. 



il.lla) 



/^r)*f 



I m \ 2m 



t = ,\ 




■■^^-''/t- 



The solution (7.76) can be written then 

-rj I, 



y e 



Jo cos 5 



-cos 



^- 



r/u-5 



(7.77b) 



(7.78) 



It is convenient to introduce the nondimensional initial velocity X= y\l {y^ 0)^) so 
that the angle (5 can be written 



S - arctan 



"^ y'oKyoCOo)-''s/(i>o ^ 



: arctan 



r ^ 

A + n 



i' 



V V d J 



(7.79) 



This writing reflects the three cases in Eq. (7.71): for r^ = co^/ cOq = (> 1, = 1, < 1) 
we have /^ - 4 m k = {> 0, = 0, < 0), respectively. The case r^ = 1 is referred to as 
critical damping, and the case I'j > 1 is called overdamping. For the cases r^ = 1 
and I'j > 1, the use of Eq. (7.71) is the best way to find nondimensional solutions 
in correspondence to Eq. (7.79). 

Damping Effect. Equation (7.78) can be used to see the damping effect, which 
is reflected by r^. We can use Eq. (7.78) for the following observations. The most 
important damping effect is that the amplitude of oscillations decreases in time. A 
second effect is that the effective frequency becomes smaller (i.e., the period of 
oscillations becomes larger) due to the appearance of the square root in the cosine 
function. A third effect is the modification of the angle S. Figure 7.4 illustrates the 
damping effects. Here, /I = is assumed for simplicity such that the solution y I y^ 
does only depend on r^. The amplitude reduction for 0<r^< 1 may be clearly seen. 
The fact that damping increases the period of oscillations is clearly visible for the 
rj = 0.5 case. For r^ = 1, the solution implied by Eq. (7.71) reads 



y_ 
yo 






= e 



{\ + {X + r,)t,) = e-"[\ + {\ + X)t,). 



(7.80) 



For this case, exp(-r^ ?,)/cos <? cannot be shown: we have d= 7il2&o that cos (?=0. 
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Fig. 7.4. The damped harmonic oscillator. The solid lines represent the solutions 
(7.78) for the r^ damping values given in the figures. The dashed lines show the 
amplitude exp(-rj t,) I cos 5. 



lA Population Ecology: Growth and Self-Limitation 



Let us switch now to the modeling of processes in population ecology. We will 
consider the evolution of a certain population in time. A population is seen as a 
group of individuals of the same species that live together in an area of sufficient 
size. The population will be measured in terms of the population density P, which 
is the number of individuals per area considered. All individuals in the population 
are considered to be identical, this means we do not account for age, sex, size, or 
other factors. For simplicity, we do not consider any immigration or emigration 
(which would modify the equations considered below by source and sink rates). 
We also do not consider any delayed responses that may lead to oscillations (such 
scenarios will be considered in Sect. 7.5). We also do not consider variations of 
the population density P in space. The treatment of the latter case would require 
the use of more advanced mathematical equations (partial differential equations). 
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7.4.1 Growth and Self-Limitation 

Population dynamics can be affected by a lot of factors. Thus, the modeling of 
the evolution of populations is usually much more challenging than the modeling 
of mechanical processes considered before. Let us consider first some typical 
scenarios to illustrate the main questions. The focus here is on the representation 
of the modeling approach, which means the question of whether or not it is possi- 
ble to postulate laws for population ecology. 

Exponential Growth. A first case is given if a population lives under optimal 
conditions, this means there is no limitation of food. For this case, the population 
dynamics are controlled by birth and death rates, which will be proportional to the 
population density P. The evolution of P in time t can be described by the conser- 
vation equation 

— = bP-dP = (h-d)P = rP. (7.81) 

dt 

Here, h and d are constant parameters that specify the birth rate b P and death rate 
d P, respectively. The difference r = b - dh called the rate of growth or decline, 
depending on whether r is positive or negative. The solution to this equation can 
be derived by adopting the solution of the differential equation (7.7). The latter 
requires the replacement ofy hy P,-\l Thy r, and the setting j^, = 0. According to 
Eq. (7.14), the solution of Eq. (7.81) is given by the expression 

P = Poe', (7.82) 

where Pq is written instead of y^. The suitability of this solution can be seen by 
proving first that the initial condition is correctly recovered: by setting ? = we 
see that P(0) = Pq as required. Next, we differentiate this solution to check that 
this function satisfies the differential equation (7.81), 

^ = rP,e"=,'P. (7.83) 

dt 

The last expression follows from the use of the solution (7.82) to replace the 
exponential function. The latter equation corresponds to the differential equation 
(7.81), which means that the function (7.82) is indeed the solution of Eq. (7.81). 
The population model (7.82) is named after the Reverend Thomas Malthus, who 
authored "An Essay on the Principle of Population" (1798), one of the earliest and 
most influential books on population dynamics. In correspondence to Newton's 
Laws of Mechanics, the Exponential Law (7.82), which is called the Malthusian 
Law, is regarded in the field of population ecology as the first principle of popula- 
tion dynamics. 
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Self-Limitation. The Malthusian Law model is only applicable to the case that 
there is no limitation of food. Obviously, this assumption is incorrect in general, 
this means the development of the population density in time may be limited by a 
maximal value. There is, therefore, the condition 

limP{t)=K. (7.84) 

Here, the constant K is called the carrying capacity. This parameter determines the 
maximum population for which a sufficient amount of food is available in a given 
area (for example, the maximum number of buffalo for which grass and water was 
available in Northern America before 1800). The fact that the population density 
approaches the constant K is referred to as self-limitation: the population growth 
leads to a self-limitation of the population because of finite resources. How can 
we model the population dynamics so that P{t) approaches K asymptotically? To 
have a higher flexibility (and to prepare applications to the modeling of the world 
population in Sect. 7.4.3), we will introduce in addition to the constant upper 
bound K a constant lower bound L = /*(-<») that refers to the population density P 
for t — > -00. The function P - L will continuously increase from zero to K - L. 
The behavior ofP-L can be described, therefore, by the relation 

P-L = (K-L)G(t). (7.85) 

The unknown function G{t) refers to a distribution function (see Chap. 4), i.e., 
G{t) is a non-negative function that increases monotonically from zero to one, 

0<G(t)<G{t + At)<l, (7.86) 

where At is any non-negative time interval. The differentiation of Eq. (7.85) by 
t provides then for P the differential equation 

^ = {K-L)g{t). (7.87) 

at 

Here, g{t) = dG{t) I dt represents the population density function (PODF) related to 
the distribution function G{t). We may assume that g{t) has the properties of a 
probability density function (PDF). The problem to be solved now is to provide a 
model for g{t). It is obvious that g{t) can be modeled in many different ways. 



7.4.2 Population Density Function Models 

Several possibilities to provide models for g(?) will be presented in this section. 
The approach applied is to consider two self-limitation models. The distribution 
function G{t) and PODF g{t) that are implied by these population models will be 
derived as a consequence of these population models. 
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Logistic Model. First, let us consider the logistic growth model introduced by 
Verhulst (1838). This population model is described and used for applications in 
several texts (see, e.g., Edelstein-Keshet 2005, Allen 2007, Murray 2002, Boyce 
& DiPrima 2009, and Turchin 2003). The logistic model reads 

'' ^^-V-Z) = ifl-^(P-Z). (7.88) 



dt tK-L t\ K-L 

Here, r is constant time scale. L = P(-oo) refers to any constant initial value, and 
K = P(co) refers to the constant carrying capacity. The term P - L extends P in the 
exponential model (7.81) by the consideration of a nonzero L, and the expression 
{K- P)l {{K-L) r] corresponds to the growth rate r in the exponential model. The 
idea of using the latter growth rate is to account for limitations of food due to an 
increasing competition for food. For small P « K, the growth rate is constant 
such that the population increases exponentially. For the case that P approaches K 
we find that the growth rate becomes zero, which means that the population levels 
off The maximum rate of change dP I dt can be found by considering the right- 
hand side of Eq. (7.88) as a function of P, 

1 K-P 
f{P) = ----iP-L). (7.89) 

T K - L 

By taking the first two derivatives we find 

(7.90a) 



df IK-P-{P-L) 
dP T K-L 


IK + L-2P 
T K-L 


dV 1 2 
dP^ T K-L 





(7.90b) 

Therefore, dP I dt has a maximum at P = (K + L) / 2. The reason why Verhulst 
called his population model the logistic growth model is not fully clear. The term 
logistics means the delivery of required goods, at a required place, at a required 
time, to the required person. Possibly, Verhulst's idea of calling this equation the 
logistic population model was that this equation accounts for logistics issues (the 
requirement for food). 

Logistic Model Solution. How can we determine the solution of the logistic 
model? We apply again the method of separation of variables, which requires that 
all variables P appear on one side, and all variables t appear on the other side, 

jK-DdP d^ 

' {K-P){P-L) ' T 

To enable the integration of the left-hand side of this relation, we have to use 
partial fraction expansion, this means we consider the term {K - P) (P - L) as the 
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common denominator of two contributions, 

1 A B _A{P-L) + B{K-P) 



{K-P){P-L) K-P P-L 



(K-P)iP-L) 



(7.92) 



Here, A and B are any unknown constants, and the last expression results from 
considering the common denominator of the previous expression. The first and the 
last expression are consistent if ^ ^B ^ 1/{K- L), which means that 



K-L 



1 1 

- + - 



(K-P)(P-L) K-P P-L 
The use of this relation in Eq. (7.91) leads to 



fl^^ ' 



K-P P-L) ' T 



dt 



This equation can be integrated, 
InlP -L\- InlP -K\ = \n^~^ 



P-K T 
where C is any constant of integration. The latter expression can be written 

P-L c „r 

■ e e . 



P-K 
To get rid of the absolute value we write 

P-L C tiT tiv 

- ±e e =ce , 



(7.93) 



(7.94) 



(7.95) 



(7.96) 



(7.97) 



P-K 
where ± e*^ was replaced by another unknown constant c. The solution reads then 



L-Kce 

\-ce"' 



K-Le'"'lc K + L{\-e'"' lc-\) , K-L 

- = L + - 



1-e I c 



To determine c we consider this relation at any ?o, 

P,-L 
where Pq = /'(?o)- Thus, c ' is given by 



K-L 
P,-L 



'o'r 



Therefore, the solution (7.98) reads 



P = L + {K-L) 



K-L 
Po-L 



,-(t-to}'r 



(7.98) 
(7.99) 

(7.100) 

(7.101) 
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The latter solution can also be written 



K+L K-L 

2 2 


+ {K-L) 


1 + ^e 
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K + L K-L P^-L 
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K-P, 
Po-L 


_g-('-'0)/!- 





(7.102) 



Centered Logistic Function. A problem related to the use of the logistic 
function (7.102) is the need to provide the asymptotic value K, which is usually 
unknown. Instead of providing K, it is often more appropriate to use knowledge 
about the point P^ = P{tc), where dPI dt has a maximum (see e.g., the discussion in 
Sect. 7.4.3). The discussion of the logistic model (7.88) resulted in the relation 
P^= (K + L) 1 2, this means the setting of P^ determines K. To use the assumed 
knowledge of P^, we replace ?o by ?(. and Pq by P^. = (AT + Z) / 2 in Eq. (7 . 1 02) - we 
can use any reference point Pq- By using the definition of P^ we find for the ratios 
involved in Eq. (7.102) the expressions 



K + L 



K-L K+L 



-L = P-L, 



K-P^ K-(K + L)/2 



P-L 



= 1. 



{K + L)I2-L 

(7.103) 
The combination of the solution (7.102) with the latter expressions leads to 



P = P, + (P,-Z)^^,J^ = P,+(P,-Z)^ 



(t-t,)l(2T) _ -(t-t,)l(2i) 



1 + e" 



)/(2r) _|_^-(M,)/(2r) 



P, +(P, -i)tanh 



2t 



:Pil + ^^^^tanh[' ^- 



2t 



(7.104) 



The advantage of this centered logistic function is that this model can be easily 
applied: for given t^ and P^ values, it just needs the adjustment of the time scale r 
to data. The derivative of P, which is required for the calculation of the PODF g, 
is given by 



dP_ 
dt 



P-L 



1 



K-L 



1 



2t ,-'{t-tc 
cosh* 

2r 



'' cosh^l^^ 

2r 



(7.105) 



Here, P^- L = {K- L)I2 was used in the last expression. 
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Population Density Function: Logistic Model. The PODF g{t) for the logistic 
model can be obtained by comparing Eq. (7.105) with dP/dt ^{K- L)g, 

g{t) = j ) Y = T 7 ^• (^-106) 



cosh 1 + sinh' 



2t J y 2t 

This function corresponds to the logistic PDF (the name of the logistic PDF arises 
from the fact that its distribution function is the logistic function). The mean and 
variance of the PODF (7.106) can be found by integrating the PODF g{t) multi- 
plied with t - 1^ and {t - t^Y, respectively, over t from negative infinity to positive 
infinity. This calculation reveals that the mean is equal to t„ and the standard 
deviation is z";r/3"^: see exercise 7.4.1. 

Kapitza's Model. Kapitza (1996) suggested a slightly different version of the 
population model (7.104) given by replacing the hyperbolic tangent function by an 
arctangent function multiplied with 2 / ;r, 



P = P 



1 H — arctan 



2r 



(7.107) 



The reason for involving the factor 2 / ;r is to account for the asymptotic behavior 
of the arctangent function. This function multiplied by 2 / ;r approaches one for 
large t as does the hyperbolic tangent function. Thus, we get P(oo) ^2P^- L^ K, 
as required according to the definition of K. Justification for Kapitza's model 
(7.107) arises from the fact that the model (7.107) is consistent with the view of 
population dynamics as a self-similar process (Kapitza 1996). By using the rela- 
tion P^-Z ^(K-L)/2, the change (iP/iinmpliedby Eq. (7.107) is given by 

^^^^^ _ = ^ZA^ _. (7.108) 

dt nx l + [(?-?J/(2r)]' 2;rr 1 + [(?-? J /(2r)] ' 

Population Density Function: Kapitza's Model. The combination of the de- 
rivative (7. 108) with the definition dPldt^(K-L)g of the PODF g(t) shows that 
the PODF for Kapitza's model is given by 

&(0 = T^— ^ ^ TV- (7-109) 

This is the structure of a Cauchy PDF. The mean and standard deviation can be 
calculated as for the logistic PODF. This calculation shows that the mean of the 
Cauchy PODF does not exist. The variance or higher-order central moments take 
reference to the mean. Therefore, these moments are not defined, too. The second 
moment about the center t^ is infinite: see exercise 7.4.2. 
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7.4.3 Application to World Population Modeling 

Let us consider the modeling of the world population development to illustrate 
the suitability of the concepts developed in Sects. 7.4.1 and 7.4.2. Table 7.2 and 
Fig. 7.5 show the corresponding data according to the Decennial Censuses, U.S. 
Census Bureau, U.S. Dept. of Commerce (World Almanac 2010). 

Logistic Model. The performance of the logistic population model (7.104), 



P = P. 



1 + ^ tanh 



(7.110) 



is shown in Fig. 7.5. The model parameters applied are given by 

t^=l999, P^=6, L = l, r = 30.65. (7.111) 

This model performs very well. The procedure to optimize the model parameters 
was the following: The value Z = 1 is in consistency with the first data value for 
t = 1804. It is not immediately obvious which t^ (and related P^) should be used to 
characterize the position of the strongest change dP I dt\ there are three reasonable 
candidates for t^ given by 1987, 1999, and 2009. A good approach for finding the 
best choice for t^ is to consider for each potential t^ (and related PJ the least- 
squares error, which reads for our case 

E'=^hP.-PMit,)f- (7.112) 

iV /=i 

Here, ?, and P, represent the data values given in Table 7.2, A'^ = 9 is the number of 
samples, and P^i refers to the logistic model (7. 1 10) considered. In particular, it is 
helpful to consider the square root of the normalized error E, = El < P^>^'^. Here, 
<p2>i/2 j-gfgjs to the standard deviation of population data given in Table 7.2. 
This normalized error E, is shown in Fig. 7.6 for the three potential t^ values in 
dependence on the time scale r. It may be seen that E, has a minimum E, = 0.0337 
at r = 30.65 for the case t^ = 1999, for which P^ = 6. The latter finding leads to the 
parameter values given in Eq. (7.1 1 1). 

Kapitza's Model. The corresponding performance of Kapitza's model (7.107), 



-L2 . t- 
1 - 



P = P\\ + s arctan 

P, TT { 2t 



(7.113) 



is also shown in Fig. 7.5, where the model parameters are given by 

<, =1999, P^=6, Z= 0.2575, r = 20.08. (7.114) 

Figure 7.5 shows that the performance of Kapitza's population model is compa- 
rable to the performance of the logistic model, this means both models perform 
equally well. The parameters for Kapitza's model were found in the following way. 
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Table 7.2. The development of the world population in time t from 1804-2050 according to the 
Decennial Censuses, U.S. Census Bureau, U.S. Dept. of Commerce (World Almanac 2010). The 
population P is measured in 10'. The population values after 2009 are projections. 
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Fig. 7.5. The dots represent the world population data according to Table 7.2. 
(a) The solid line shows the prediction of the logistic model (7.110); (b) the solid 
line shows the prediction of Kapitza's model (7.113). 
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Fig. 7.6. The normalized least-squares error E, = E I < P^ >^^^ for three potential t^ 
values in dependence on the time scale r. The upper pictures apply to the logistic 
model (7.1 10). The lower pictures apply to Kapitza's model (7.1 13). 
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Fig. 7.7. PODFs g{t) multiplied by 100 for two population models. The solid lines 
show in (a) the PODF (7.106) of the logistic model and in (b) the PODF (7.109) of 
Kapitza's model. The dashed lines show the normal PODF (7.117) that has the 
same mean and variance as the logistic PODF. 



In contrast to the behavior of the logistic model, Fig. 7.5 shows that Kapitza's 
model provides population changes for t < 1850. Hence, the direct setting of any 
L = P(-co) in Kapitza's model appears to be inappropriate. A way to account for 
this problem is to consider Kapitza's model at a data point (?,, Pi), 



P.=R 



1 + 



P-L 2 



■ arctan 



It 



(7.115) 



By adopting an appropriate data point (?,, Pi) of the data given in Table 7.2, the 
latter relation gives the opportunity to calculate L, 



L = P^ 



P-P 



arctan 



It 



(7.116) 



This approach was used to calculate E, depending on t for the three potential t^ 
values, where (ty. Pi) = (1804, 1) was used. Figure 7.6 shows that E, has a mini- 
mum E, = 0.0382 at r = 20.08 for the case that t^ = 1999 and P, = 6. The latter 
observation implies the parameter values given in Eq. (7.1 14). 

Population Density Functions. The PODFs (7.106) and (7.109), implied by 
the logistic model and Kapitza's model, respectively, are shown in Fig. 7.7, where 
the corresponding parameters (7. Ill) and (7. 114) are used. A PODF 



git)- 



1 



lln a 



-exp 






(7.117) 



that has the structure of a normal PDF is also shown for a comparison. Here, 
a= TTilV^' according to the standard deviation of the logistic PODF. Clearly, the 
logistic PODF is similar to the normal PODF: the biggest difference is the higher 
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peak value of the logistic PODF (which is related to the flatness m^^ll / 5 = 4.2). 
On the other hand, the PODF of Kapitza's model reveals significant deviations 
from the normal PODF. The most important difference to the normal PODF is the 
very slow decay of the density function. This difference explains the fact that the 
Cauchy PODF implied by Kapitza's model has a nonexisting mean and central 
moments: the corresponding integrals cannot exist because the PODF decays too 
slowly for large deviations from the center t^. The existence of the mean and 
central moments is not a required property for PDFs and PODFs, but it is clearly a 
desired property. 



7.5 Population Ecology: Oscillations and Collapse 

Growth and self-limitation discussed in Sect. 7.4 represent two basic features of 
population dynamics, but there may be also other basic processes that have to be 
considered. The purpose of the discussion in this section is to present a more 
general picture of population changes. On this basis, we will consider the question 
of which trend will possibly be seen regarding the future development of the 
world population. 





Fig. 7.8. Basic scenarios of population dynamics: (a) continuous growth, (b) 
balance, (c) oscillation, and (d) collapse. The solid lines present the population 
density P, and the dashed lines present the corresponding carrying capacity K (see 
the explanations in Sect. 7.5.3). 
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7.5.1 Basic Population Dynamics Scenarios 

Figure 7.8 illustrates basic possibilities (Meadows et al. 2004) in which a popu- 
lation can interact with its asymptotic limit, the carrying capacity. Let us have a 
closer look at these scenarios in order to prepare the following discussions in this 
section. 

Case 1 and Case 2. A case 1 illustrated in Fig. 7.8a, which was considered in 
Sect. 7.4, is given by a population that does continuously grow if the carrying 
capacity is much higher than the population values, or if the carrying capacity in- 
creases faster than the population. A case 2 illustrated in Fig. 7.8b, which was also 
considered in Sect. 7.4, is given by a population that levels off smoothly below the 
carrying capacity. Examples for these two cases were given by the exponential 
increase of the U.S. population from 1790 to 1890 discussed in Chap. 1 and the 
world population growth discussed in Sect. 7.4. 

Case 3 and Case 4. There are also other population dynamics scenarios that 
may be observed if the population overshoots the carrying capacity. Overshooting 
may occur if there are delayed responses involved such that the population does 
not immediately experience the limiting action of the carrying capacity. A case 3 
illustrated in Fig. 7.8c is given if the population overshoots the carrying capacity 
without doing massive and permanent damage to the system. In such a case, the 
population will oscillate about the carrying capacity and level off after estab- 
lishing a new balance with the carrying capacity. Another possibility is a case 4 
illustrated in Fig. 7.8d: the population does overshoot the carrying capacity with 
severe and permanent damage of system resources. In such a case, the population 
will rapidly decline to achieve a new balance with the reduced carrying capacity at 
a much lower level. This case will imply a collapse of the original system. The 
term collapse used here does not mean the extinction of a population, but it is used 
to refer to the fact that the original system (characterized by the original constant 
carrying capacity) does not exist anymore. An example for these cases is the 
following: deer or goats, when natural enemies are absent, often overgraze their 
range, which causes erosion or destruction of the vegetation (Kormondy 1996). 
The consequences of such overgrazing are usually seen with a typical delay time 
of several months. 

Future of Mankind. An obvious question regarding the development of the 
world population is whether and under which conditions the world population and 
related factors will grow, hold steady, oscillate, or collapse in the future: which 
out of the four scenarios described in the preceding paragraph will be realized? 
The latter question will be addressed in Sects. 7.5.2 and 7.5.3 by presenting two 
population models and their conclusions: the World3 model and a logistic model 
that can account for delay effects. 
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7.5.2 The WorldS Model 

World3 Model. In 1968, thirty individuals from ten countries (scientists, 
educators, economists, humanists, industrialists, and international civil servants) 
gathered in Rome to discuss a subject of stagerring scope: the present and future 
predicament of mankind. The purpose of this meeting was to examine the "world 
problematique": poverty in the midst of plenty, degradation of the environment, 
loss of faith in institutions, uncontrolled urban spread, insecurity of employment, 
alienation of youth, rejection of traditional values, and inflation and other mone- 
tary disruptions (Meadows et al. 1972). Out of this meeting in 1968 grew the Club 
of Rome, which is an informal organization. Its purposes are to foster under- 
standing of the global economic system in order to bring that new understanding 
to the attention of policy-makers and the public worldwide, and to promote in this 
way new policy initiatives and actions. The Club of Rome commissioned a team 
of analysts from the Massachusetts Institute of Technology (MIT) to analyze the 
"world problematique" using a computer model called WorldS developed at MIT. 
The time scale for the model began in the year 1900 and continues until 2100. 
Historical values to the year 1970 are broadly reproduced in the Worlds output. 
The results of this analysis were published for the general public by Meadows et 
al. (1972). A detailed description of the World3 model is available in the book 
"Dynamics of Growth in a Finite World" by Meadows et al. (1974a). A 30-year 
update of the 1972-analysis was published by Meadows et al. (2004). Recently, 
Turner (2008) compared the World3 model output (Meadows et al. 1974b) with 
30 years of reality (for 1970-2000). The following presentation in this subsection 
follows Turner's (2008) analysis. 

World3 Model Structure and Variables. The World3 model involves four 
key elements. A first key element is the presence of resources, such as agricultural 
land that may be eroded as a result of the functioning of the economic system. 
A second key element is the presence of delays in the signals from one part of the 
system to another. For example, the effect of increasing pollution levels (see the 
discussion of global warming in Chap. 1), may not be recognized for some dec- 
ades. A third key element is the consideration of both positive and negative feed- 
back loops. The interaction of such feedback loops may lead to oscillations of 
variables over time. The fourth key element is given by the consideration of the 
global economic system as a complete system of several subsystems: population, 
food production, industrial production, consumption of nonrenewable natural 
resources, and global pollution. A detailed explanation of the meaning of these 
model variables is given in Turner's (2008) review "A comparison of The Limits 
to Growth with 30 years of reality". 
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Fig. 7.9. The Worlds results. The solid lines present the standard scenario World3 
results, the hatched lines present the comprehensive technology scenario results, 
and the dashed lines present the stabilized world scenario results (Meadows et al. 
1974b). Historical values to the year 1970 are reproduced by all scenarios with the 
exception of nonrenewable resources of the comprehensive technology scenario. 
For this case, the natural resource reserves were assumed to be twice as high as for 
the other cases. The dots present observed data for the period 1970-2000 
according to Turner's (2008) analysis. The observed data were normalized to the 
Worlds model values for 1970. Regarding the nonrenewable resources, the lower 
dots and upper dots provide minimum and maximum estimates. The global COt 
concentration was used to obtain the global pollution curves. 



World3 Model Scenarios. The analysis of Meadows et al. (1974b) considered 
the following three basic scenarios that are illustrated in Fig. 7.9: 
• standard scenario: 
This scenario assumes a business-as-usual situation. Parameters reflecting phys- 
ical, economic, and social relationships are maintained at values consistent with 
the period 1900-1970. This scenario results in "overshoot and collapse" of the 
global system (see the case 3 scenario of Fig. 7.8) about midway through the 
2 P' century due to a combination of diminishing resources and increasing eco- 
logical damage due to pollution. 
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• comprehensive technology scenario: 

This approach attempts to solve sustainabUity issues with a broad range of pure- 
ly technological solutions. The scenario considers levels of resources that are 
effectively unlimited. It is assumed that 75% of materials are recycled, pollution 
generation is reduced to 25% of its 1970 value, agricultural land yields are dou- 
bled, and birth control is available worldwide. These efforts delay the collapse 
(see the case 3 scenario of Fig. 7.8) of the global system to the later part of the 
21"' century, when the growth in economic activity has outstripped the gains in 
efficiency and pollution control. 

• stabilized world scenario: 

Both technological solutions and deliberate social policies are implemented to 

achieve equilibrium states for key factors including population, material wealth, 

food, and services per capita (the case 2 scenario of Fig. 7.8). Such equilibrium 

states may be achieved, for example, by perfect birth control, a pollution control 

technology, and a maintenance of agricultural land. The variables in Fig. 7.9 

approach equilibrium values in this case (except the nonrenewable resources). 

Model Evaluation. Turner (2008) compared World3 predictions with 30 years 

of reality (which means with observed data for 1970-2000). He concluded that 

these observations compare favorably with key features of the standard scenario 

(business-as-usual) predicting "overshoot and collapse" of the global system about 

midway through the 2V' century. The data do not compare well with the other two 

scenarios (the comprehensive technology and stabilized world scenarios). 



7.5.3 A Delay Logistic Model 

Delay Logistic Model. The World3 model is very complex such that it cannot 
be easily studied. Let us cover some basic assumptions of this model on the basis 
of a much simpler model. In particular, let us consider the following extension of 
the logistic model (7.88), 

This delay logistic model involves a memory term 

M{t)=\/j{t-s){P{s)-L)ds. (7.119) 

— GC 

The function ju{t - s) is a memory function: all the population history is used for 
the calculation of the change dPI dt at t. The memory effect also represents a delay 
effect: the effect of the population P{s) with -co < s <t on dPI dt{t) can be seen as 
a delayed effect. The model (7.118) is unclosed as long as the memory function 



7.5 Population Ecology: Oscillations and Collapse 281 

ju{t-s) is not defined. The carrying capacity will be considered as a function of t 
here, K = ^^(0, to allow dynamic interactions between the population density P 
and K. The model parameter L represents again any constant initial population 
level, L = P{-co), and the characteristic time scale r is a constant, too. 

Definition Memory Function. The memory function can be defined in many 
different ways (Allen 2007). Here, we consider the model 

^(t-s) = —exp\-'^-^^^\, (7.120) 

^M [ ^M ] 

where t,^ refers to a characteristic memory time scale. This memory functions has 
the normalization property to integrate to one, 

j /j{t -s)ds = — I exp<^ yds = expl \ - exp< [ = 1 . 

-on t fl,f -CD \ ^ AA \ \ ^ A/f \ t A/ 






(7.121) 
How are the logistic model (7.88) and the delay logistic model (7.118) combined 
with Eq. (7.120) related to each other? The consideration of the memory term in 
the limit r^ — > shows the following, 

lim M(t)= f {P{s)-L) lim /j(t-s)ds 

(7.122) 
= (P(0 -L)\ hm /j{t -s)ds= P{t) - L. 

The conclusion that P{s) - L can be replaced by P{i) - L can be justified in terms 
of the properties of the memory function: ju(t-s) is always zero for Zi/ — >-0 except 
at ^ = r (i.e., ju{t - s) behaves like a delta function). Therefore, only P{t) - L will 
contribute to the integral. The last expression in Eq. (7.122) arises from the nor- 
malization property (7.121), which applies to all r^^. Thus, we can conclude that 
the delay logistic model (7.118) combined with Eq. (7.120) recovers the logistic 
model (7.88) in the limit r^^ 0. 

Model Reformulation. The analysis and application of the model (7.118) are 
complicated because of the memory termM(?). However, the use of the properties 
of the memory function ju{t-s) enables a much simpler representation of the delay 
logistic model. First, we differentiate M(t) to obtain for M the equation 

^=\^'^{p(s)-L)ds.MO){P(t)-L) 

dt _^, dt 
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Table 7.3 Parameters of the model (7.124) for the curves shown in Fig. 7.10. 
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where the definitions of M and ju(t-s) are applied. By making use of this equation 
for M we can write the model (7. 1 1 8) in terms of an equation system for P and M, 

dM P-L-M ..,,.„ ^ 
= . (7.124b) 

dt T^ 

Equation (7.124b) enables the derivation of the conclusion ij ATI) for r^^ — > in a 
simple way: by multiplying this equation with r^^ and using r^^ — > we recover 
M= P - L. The equation system (7.124) has to be combined with appropriate 
initial values at any time ? = 0, 

^0=^(0), (7.125a) 



Mo =M(0)= \/j{-s){P{s)-L)ds. (7.125b) 

Relation (7.125b) shows that the previous history of P(.s') with -co < s <() has to be 
provided to calculate Mo. We will assume here that P{s) = Pg for -oo < s < 0. 
For this case, Mo is given by 

M,={P,-L)]Mi-s)ds=P,-L. (7.126) 

The last expression follows from Eq. (7.121) by setting ? = 0. Unfortunately, it is 
impossible to solve the equation system (7.124) analytically, i.e., these equations 
have to be solved numerically. 

Four Scenarios. The population model (7.124) is capable of covering the four 
basic scenarios of population dynamics shown in Fig. 7.8. This will be made clear 
in the following by using four definitions (corresponding to the four scenarios 
considered) for the carrying capacity K{t). The four modeling approaches applied 
for K{t) will be explained in terms of Fig. 7.10. This figure agrees with Fig. 7.8: 
the difference is that the model parameter values (which are given in Table 7.3) 
and function values are specified in Fig. 7. 10. 
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Fig. 7.10. Basic scenarios of population dynamics: (a) case 1: continuous growth, 
(b) case 2: balance, (c) case 3: oscillation, and (d) case 4: collapse. The solid lines 
show the population density P, and the dashed lines show the carrying capacity K. 
The numbers applied here are arbitrary (an initial carrying capacity Kg^ I was as- 
sumed). The model parameters are given in Table 7.3. 



Case 1: The first case assumes an infinite value ofK, 
i:^ ->oo. 



(7.127) 



Correspondingly, the population model (7.124a) is equivalent to the use of the 
exponential population growth model 



dP^ 

dt 



P-L 



(7.128) 



This model was used to obtain Fig. 7.10a that shows an exponential increase. 
• Case 2: The second case assumes a constant and finite value of i^. 



K=K, 



(7.129) 



Here, K^ refers to the initial value ofK. This case corresponds to the consideration 
of a logistic model if there are no memory effects, i.e., if Tg^f = 0. This model was 
applied to obtain Fig. 7.10b (this case can be handled numerically by replacing 
Eq. (7.124b) for the calculation of M by M = P - Z, see the discussion below 
Eq. (7.124)). Another case 2M (the M refers to the inclusion of memory effects) is 
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given if t^ is nonzero. The case 2M enables the simulation of oscillations about 
the carrying capacity K. Finally, a stationary state is established where 



»4l- 



M 



K-L 



(P-L), 



= 



P-L-M 



(7.130a) 



(7.130b) 



This stationary state is realized by the values M = K - L and M = P - L, which 
implies that P = K. An interesting question is under which condition the logistic- 
shape behavior of curves changes such that P overshoots the carrying capacity K. 
The latter question is addressed in terms of Fig. 7. 11 that illustrates the effect of 
the memory time scale t^. We see that t^,I rhas to be sufficiently large to enable 
an overshooting. The overshooting is observed if r^i > r/ 3: in this case we find 
that {P - K)l K> 10 '. Figure 7.11a presents P{t) for the critical value r^f = r/ 3, 
and Figure 7.11b shows the overshooting effect for t^ = 3 r/2. Interestingly, the 
critical value t^= tI 3 was also found for variations of model parameters (e.g., 
other T, Pq = Mq, and Kq values than applied for the cases here). 
• Case 3 : The third case assumes that K changes in interaction with P, 



dK 

dt 



-9{t-t^) 



K-P 



(7.131) 



Here, t^^ is a characteristic time scale for changes of K, and ?^- is the smallest time 
for which P(tf^) = K. The idea of this model is that K begins to decrease if P over- 
shoots the carrying capacity, P > K. According to this model, it is possible that K 
increases at later times if K > P (which models a recovery of the carrying capac- 
ity), see Fig. 7.10c. 
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Fig. 7.11. The solid lines present the case 2M scenario, where L ^ 0, Pq ^ Mq 
= 0.1, Kq = 1, and r= 25. The difference between these lines is the use of t,,/ = r/3 
and Tm^3 tI 2 m (a) and (b), respectively. The dashed lines show the correspond- 
ing case 2, where % = (see Table 7.3); the hatched lines show K^Kq^ 1 . 
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Table 7.4 Parameters of the model (7.124) for the curves shown in Fig. 7.12. 


L Po Afo Ko T Tu 


Tk 


Case 2 1 1.0151 0.0151 11 30.65 
Case2M 1 1.0151 0.0151 10.52 30.95 4.40 






Case 4: The fourth case also assumes that K changes in interaction with P, 

(7.132) 



dK ' - "^ 
= mm 

dt 



0,^^ 



Here, K decreases as in the case 3 model '\f K> P, but an increase of AT is impos- 
sible {K does not change iiK > P), see Fig. 7. lOd. 

World Population Modeling. Let us apply the population model (7.124) to the 
modeling of the world population. The population data considered are given in 
Table 7.2. Obviously, the case 1 modehng approach of infinite growth does not 
apply to this data set. For the case considered we have P < K. This fact suggests 
the use of the case 2 (for which v^ = 0) or case 2M (for which v^ # 0) models that 
apply a constant K, this means it is expected that K does not change due to an 
overshooting of P. It was proven that the case 3 and case 4 modeling approaches 
do not represent an alternative to the use of the case 2 and case 2M approaches. 
The case 2 model parameters correspond to the data used for the logistic model 
(7.110) combined with Eq. (7. 111). Here, P^ = P(t = 1 800), where P(t) is given by 
Eq. (7.1 10). The curve for the case 2 model agrees exactly with the logistic model 
(7.1 10). The parameters for the case 2M model were obtained by keeping M, Pq, 
and Mo = Pq- L fixed and optimizing K^, r, and r^i such that the least-square error 
becomes minimal. A way to realize this nontrivial optimization problem is to 
optimize K^ for given r and r^/ values, and to consider then the effect of r and r^ 
variations. The normalized error E, = 0.0271 that results from this optimization of 
parameters is smaller than the corresponding normalized error is, = 0.0337 related 
to the logistic curve. Hence, the case 2M model is slightly more accurate than the 
case 2 model. Figure 7.12 shows a comparison of the case 2 model and case 2M 
model. The corresponding model parameters are given in Table 7.4. There is 
hardly any observable difference between the two models. The latter comparison 
confirms the World3 assumption about a nonzero memory time t,^,. However, the 
small memory time r^ = 4.4 years obtained here means that the basic feature of 
the logistic curve to approach K smoothly (so that P < AT) is unchanged. Accord- 
ing to the discussion of the case 2M above, it would need r^i > r/ 3 to enable 
values P> K. Hence, this analysis does not support the case 4 scenario features 
that are obtained by the World3 model (see Figs. 7.8 and 7.9). 
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Fig. 7.12. World population modeling based on the model (7.124). The solid lines 
in (a) and (b) show the result of using the case 2 model and the case 2M model, 
respectively. The model parameters for these cases are given in Table 7.4. The 
hatched lines show the corresponding carrying capacity K. The dots present the 
observed data and predictions according to Table 7.2. 



7.6 Summary 



This chapter described the mathematical analysis of a variety of processes. The 
objective was to find equations that we can use for the analysis and prediction of 
processes. Actually, our ultimate goal is to find general equations (laws) that can 
be applied to a relatively broad range of processes. Did we accomplish this goal? 
Let us address the latter question by having a closer look at the properties of a law 
and the equations that we found for mechanical and population ecology processes. 

Laws. The term law is often used with different meanings. Here, the term law 
refers to a mathematical model that represents an idea about the mechanism of a 
processes considered. An important property of a law is that it has to be proven to 
be correct: there has to be convincing evidence for this law given by observations 
(e.g., measurements that can be repeated). Unfortunately, it will never be possible 
to have only one equation for everything. The reason is simply that an equation 
has to explain the mechanism of processes, and there is a huge variety of different 
processes. Consequently, we have to use four population models for four different 
population processes. There may be different (mechanical or population ecology) 
processes that look relatively similar, e.g., a simple balance of heat transfer and 
the logistic population curve. However, also for such similar processes we have to 
use different models because the nature of these processes is different. Hence, the 
goal can only be to find laws that cover certain processes. A law usually involves 
model parameters that enable the application of the model to a certain range of 
processes. 
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Laws of Mechanics. We discussed various mechanical processes: the distribu- 
tion of heat or mass, and oscillations. Regarding the distribution of heat we saw 
that there exists a theoretical framework given by Newton'sLaw of Cooling, and a 
corresponding equation can be applied to the modeling of the distribution of mass. 
Motions of macroscopic bodies (moving with velocities much smaller than the 
speed of hght) can be described very well by Newton's Laws of Motion. Thus, we 
have laws that we can use to describe mechanical processes. This does not mean 
that we have completely specified equations for every possible case. For example, 
the suitability of using a damping force that is a linear function of the velocity re- 
quires evidence (depending on the case considered it may be more appropriate to 
use other damping force models). The meaning of the conclusion that there exist 
laws for the mechanical processes considered here is that there is an established 
mathematical basis for addressing modeling problems. 

Laws of Population Ecology. We also considered the question of whether or 
not there are laws of population ecology. This question is more complicated than 
the question about laws for mechanical processes because population systems are 
often much more complex than mechanical systems. There are different opinions 
about the existence of population dynamics laws. One widespread opinion among 
ecologists is that ecology lacks general laws. On the other hand, Turchin (2001, 
2003) argues that there are some very general law-like propositions that provide 
the theoretical basis for most population dynamics models. Scheiner and Willig 
(2005, 2008) argue that the fundamental principles (similar to Darwin's (1859) 
nonmathematical theory of evolution) of ecology are known, but these principles 
are not yet manifested in mathematical models. The conclusion that can be drawn 
from the discussion here is that there is currently no law of population ecology in 
the sense that there is one equation that applies to a variety of similar processes. 
However, this situation may change. A basic element that is missing in equations 
for population dynamics is the PODF, which has all the properties of a probability 
density function (PDF). By following the modeling concepts presented in Chap. 8 
it is possible to postulate an equation for the PODF. 



7.7 Exercises 



7.2.1 A cup of coffee at 90°C is poured into a mug and left in a room at 21°C. 
After one minute, the coffee temperature is 85°C. Suppose that the coffee 
temperature does obey Newton's Law of Cooling. The coffee becomes safe 
to drink after it cools to 60°C. How long will it take before you can drink 
the coffee, this means at which time is the coffee temperature 60°C? 
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7.2.2 A cup of coffee is poured into a mug and left in a room at 15°C. After one 
minute, the coffee temperature is 80°C. After two minutes, the coffee tem- 
perature is 72°C. 

a) Use Newton's Law of Cooling for calculating the coffee temperature at 
the initial time. 

b) Determine the coffee temperature after five minutes. 

7.2.3 Newton's Law of Cooling assumes that the temperature decrease is propor- 
tional to the difference between the temperature and the constant external 
temperature. For cooling in still air, a better model is to assume that the 
temperature decrease is proportional to the 5/4* power of the difference be- 
tween the temperature and the constant external temperature. 

a) Formulate the corresponding differential equation. 

b) Calculate the solution of this differential equation by using the method 
of the separation of variables. 

c) Assume that the initial temperature is equal to the external temperature. 
Use the solution derived in b) to determine the temperature for all t. 

1.1 A Solve the coffee temperature problem described in exercise 7.2.1 by using 
the solution obtained in exercise 7.2.3. 

7.2.5 Stefan-Boltzmann's Law of Radiation states that the temperature change 
dTI dt of a body at T degrees Kelvin is proportional to ii '' - T'', this means 
that dTI dt = k{E*- 7"*). Here, E refers to the constant absolute tempera- 
ture of the surroundings (measured in degrees Kelvin), and A; is a constant. 

a) Rewrite the differential equation dT I dt = k{E'^ - T'*) by introducing a 
nondimensional temperature T, and nondimensional time t,. 

b) Determine the solution T, of the nondimensional differential equation by 
using the method of the separation of variables. Hint: use the following 
integral {a and C refer to any constants): 

r dx I , f a + x"\ 1 f x"\ ^ 

— = — -In + — -arctan — + C. 

'a^-x' 4a' {a-xj 2a' {aj 

c) Explain how the dependence of T, on t, can be graphically illustrated for 
a given value of the initial nondimensional temperature T,q. 

7.2.6 A swimming pool has a volume of 50 m^ A mass C (in kg) of chlorine is 
dissolved in the pool water. Starting at a time t = 0, water containing a con- 
centration of 0.1 C/ F chlorine is pumped into the swimming pool at a rate 
of 0.02 mV min, and the water flows out at the same rate. 

a) Present the differential equation for the chlorine mass Q(f). 

b) Find the solution Q(t) to this equation. 
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c) What is the amount of the chlorine mass Q{t) after 2 hours? 

d) At which time is the chlorine mass in the pool 50% of the initial mass? 

7.2.7 Consider the following problem (Boyce & DiPrima 2009). A swimming 
pool has a volume of 50 m^. The water has been contaminated by 5 kg of a 
nontoxic dye that leaves a swimmer's skin an unattractive green. The filter 
system can take water from the pool, remove the dye, and return the water 
to the pool at a flow rate of 0.02 m^/ min. 

a) Present the equation for the temporal development of the dye mass Q(t). 

b) Find the solution Q{t) to this equation. 

c) The effect of the dye is imperceptible if its concentration c ^ Q/ Vis less 
than 5 g / m^. How long does it take to reduce the dye concentration to 
this value? 

7.3.1 A projectile is fired with an angle of elevation a and an initial velocity 
given by Vq = Vq cos ai+ Vq sin aj. Here, / andy refer to unit vectors in the 
horizontal x direction and vertical y direction, respectively. The projectile 
motion is only affected by the gravity force F = - m g j, where m is the 
mass of the projectile and g is the gravity acceleration. Use Newton's Sec- 
ond Law to do the following: 

a) Calculate the velocity vector v{t). 

b) Calculate the position vector r{t). The initial position vector is r(0) = 0. 

c) Calculate the horizontal distance // traveled by the projectile. 

d) Calculate the value of a for which the distance //has a maximum. 

7.3.2 Consider the following initial value problem {A is any parameter): 

^ + (^ + 4)^ + (^ + 3)j. = 0, j(0) = l, 4^(0) = 0. 

dt dt dt 



a) Find the solution to this initial value problem. 

b) For which values of the parameter^ are solutions bounded as ? — > oo? 

c) Find the critical t value at which y{t) may have an extreme value. 

d) Consider the parameter A range for which we have bounded solutions. 
Does y{t) have a local minimum or maximum at this critical t value? 

7.3.3 Consider the following initial value problem {A is any parameter): 

4^ + 4 + 4, = 0, ,(0) = 1, >) = ^. 

dt dt dt 

a) Find the solution to this initial value problem. 

b) Find the critical t value at which j(?) may have an extreme value. 

c) For which values of the parameter A do we find at the critical t value 
a local minimum, a local maximum, and no extreme value ofy{t)l 
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7.3.4 Find a differential equation that has the general solution j = c^ e^' + C2e '. 

7.3.5 The displacement of a spring-mass system is described by the initial value 
problem (jq is any parameter) 

^^4±.Sy = 0, yiO) = y„ >) = 0. 

dt dt dt 



a) Find the solution to this initial value problem. 

b) Write the solution derived in a) so that the time dependence does only 
enter via an exponential function and a cosine function. Hint: follow the 
transition from Eq. (7.73) to Eq. (7.76). 

7.3.6 Consider the damped harmonic oscillator solution (7.76). 

a) Calculate the time rbetween any successive maxima. 

b) Calculate the ratio R of the displacements of any successive maxima. 

c) Explain how the damping constant ;'can be calculated via measurements 
ofR, T, and the mass m. 

7.3.7 Electrical vibrations in electric circuits can be described by the following 
differential equation for the charge Q{t) measured in coulombs (C), 

dt^ dt C ' 



1,1 1, ^^ 



Here, R refers to the resistance, which is measured in ohms (Q), L refers to 
the inductance, which is measured in henrys (H = Q s), and C refers to the 
capacitance, which is measured in farads (F = s / Q). The variables L, R, 
and C are non-negative and assumed to be constant. An impressed voltage, 
which can involve a forcing on the right-hand side of the equation, is not 
considered. The equation considered represents Kirchhoff s Second Law. 

a) Assume that Z = 2 H, 7? = 20 Q, and C = 0.01 F. Find Q(t) for the case 
that QiO) = 5 C and dQ/ dt (0) = 0. 

b) Assume that Z = 2 H and C = 0.01 F. Which R value is required so that 
the circuit is critically damped? 

c) Find the charge Q(t) for the case of a critical damping considered in b). 
Assume that Q{0) = 5 C and dQ/ dt (0) = 0. 

7.4.1 Consider the logistic PODF given by Eq. (7. 106). 

a) Calculate the mean implied by this PODF. 

b) Calculate the standard deviation implied by this PODF. 

c) Calculate the skewness and flatness implied by this PODF. 

Hint: use the following integral values, which can be found numerically. 

" x^ 1 °° x'' 7 

j r dx = —7i:^, \ r dx = n'^ . 

_„cosh (x) 6 _„cosh (x) 120 
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7.4.2 Consider Kapitza's PODF model given by Eq. (7. 109). 

a) Show that the mean implied by this PODF does not exist. 

b) Show that the second moment about the center t^ is infinite. 

7.4.3 A modification of the logistic model is given by the model of Schaefer 

dt t\ k) 

The model, which was developed for the simulation of the development of 
fish populations, is equivalent to the logistic model for E ^ 0, where L = 
/■(-co) = is assumed for simplicity. The last term -E P takes into account 
(human) predation that reduces the rate of population growth. It is reason- 
able to consider this term to be proportional to P\ the effect of predation 
will increase with the population density. The variables K, E < \ I r, and r 
are assumed to be non-negative and constant. 

a) Write the model in the form of the logistic model (the structure of this 
rewritten model will be equal to the logistic model but the parameters 
are different). 

b) Calculate the solution of this rewritten model by taking reference to the 
solution of the logistic model. 

c) Explain the effect of a nonzero E on the population dynamics in compar- 
ison to the logistic model. 

7.4.4 A modified version of the Schaefer model is given by the model 

dt y K) K\ 2 ) A 

where H is non-negative constant. The last expression provides a rewriting 
that is helpful for the following analysis. 

a) Rewrite the model by using the nondimensional variables P, = P I K and 
U^rt. 

b) Discuss the existence of equilibrium solutions for the cases H > rK/ 4, 
H^rK/4,andH<rK/4. 

c) One of the three cases considered in b) leads to the conclusion that there 
are two equilibrium points. Write the nondimensional equation in the 
form dPJdt, = -(P,i-P,)(P,2-P*) for this case, where P.i and f.j refer 
to the two equilibrium states. Use this equation to discuss dP, I dt, for 
the three cases P.q < P-i^ P*i < -f»o < P*2^ and P,2 < Aq. Here, P.q denotes 
the initial value of P.. Use this insight into the behavior of dP,ldt, to 
explain the behavior of solutions close to the equilibrium points (explain 
which of the two equilibrium points P,i and P,2 can be realized). 
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7.4.5 The model of Gompertz, which is used for the modeling of the growth of 
a tumor, is given by the equation 

dt T \P) 

Here, the variables K and rare assumed to be non-negative and constant. 

a) Calculate the solution of this differential equation. Hint: use the variable 
transformation J = \n{PI K). 

b) Calculate the equilibrium solution. 

c) Show forQ<P<K that the change dP I dt determined by the Gompertz 
model is equal or larger than dP I dt = (1 - P I K) P I ras given by the 
logistic model. Hint: you may wish to show the validity and make use of 
f(x) = 1 + Inx - X < for X > 0. 

7.4.6 The consideration of population development as a self-similar process does 
imply the following differential equation (Kapitza 1996), 

dP dt 

■■ a- 



P-P, t-t^ 

Here, P^ and t^ are the coordinates of a reference point, and « is a constant. 

a) Find the solution to this differential equation. Express the constants in 
the solution in terms of the data Pg ^nd ?o at the initial time. 

b) The exponential growth and logistic models do not describe self-similar 
processes. Does this mean that the latter models have a poor standard? 

7.5.1 Consider a generalization of the differential equation (7.7), 

-^ = -j\Kt-s)\y{s)-y^]ds, 

which includes memory effects. Here, T and y^ are constants, and /j{t - s) is 
a memory function. We assume that ju(t-s) is given by Eq. (7.120), 

/ ^ 1 [ U-' 

/u{t - s) = — exp< 

where t^ refers to a characteristic memory time scale. 

a) Write the differential equation for the limit case t^ — > 0. 

b) Consider the case that r^/ is nonzero. Show that the differential equation 
considered can be written as (follow the discussion in Sect. 7.5.3) 

d^y \ dy I y-y„ „ 

— J- + =- + — ^^ = 0. 

dt Tu dt Tf^ T 

c) Derive the limit r^^ — > of this second-order differential equation. 
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7.5.2 Consider the second-order linear differential equation derived in part b) of 
the exercise 7.5.1. 

a) Find the solution >>(?) to this equation. It is assumed that z^i < T/4. 

b) Specify the eigenvalues for the case that r^^ « T. Hint: you may wish to 
use the relation (1 ± x)"^ = 1 ± x/2 for very small x. 

c) Use the results obtained in b) to determine the solution for t^ — > 0. 

7.5.3 Consider again the differential equation discussed in exercise 7.5.1, 

^--y\Mit-s)\yis)-y^]ds. 
at T 4 

For which choice of the memory function /j{t - s) does this equation imply 
the delay differential equation 

dt T 

Here, r refers to a non-negative and constant delay time. 



8 Stochastic Evolution 



Stochastic changes were discussed in Chap. 6 by involving randomness in the 
deterministic difference equations considered in Chap. 5. This approach can be 
used to demonstrate the relevance of randomness, to show typical formulations of 
stochastic equations, and to illustrate the analysis of stochastic processes. On the 
other hand, this approach is of empirical nature and does not reveal any insight 
into the general structure of equations for stochastic processes (i.e., the question of 
what are the laws of stochastics), the conditions for the applicability of certain 
equation types, the relations between equations for stochastic processes and equa- 
tions for the PDF related to stochastic processes, and the solution of PDF equa- 
tions. The latter questions will be addressed in this chapter. This will be done by 
considering the continuous evolution of stochastic processes, which represents an 
appropriate basis for explaining the relationship between equations for stochastic 
processes and differential equations for the PDF evolution. There is a close rela- 
tion between this chapter and Chaps. 6 and 10. The difference equations described 
in Chap. 6 provide the numerical solution for the differential equations considered 
in this chapter, and the latter equations represent the basis for the stochastic differ- 
ential equations for several variables considered in Chap. 10. 

The motivation for considering the typical properties of stochastic differential 
equations and related PDF evolution equations will be explained in Sect. 8.1. Sec- 
tions 8.2 and 8.3 will address the questions considered on the basis of PDF equa- 
tions, i.e., evolution equations for PDFs and their solutions will be discussed. 
Sections 8.4 and 8.5 address the same questions on the basis of stochastic differ- 
ential equations. Here, emphasis is placed on the relations between PDF equations 
and stochastic differential equations and the choice of an appropriate stochastic 
equation for the modeling of a stochastic process considered. The basic features of 
equations for continuous stochastic processes will be summarized in Sect. 8.6. 
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Fig. 8.1. The evolution of a stochastic process X(?) in time t is illustrated in terms 
often realizations in (a). The evolution of the corresponding VDY f{x) is shown in 
(b) at ? = 10 {solid line) and t = AQ {dashed line). 



8,1 Motivation 



Stochastic Process and PDF Evolution. There are two possibilities to look at 
the evolution of a random variable and its PDF. The first approach, which was 
applied in Chap. 6, is to consider an evolution equation for the random variable. 
This approach determines the evolution of the corresponding PDF: the PDF can be 
calculated numerically at every time (see the illustration in Fig. 8.1). The sec-ond 
approach is to consider an evolution equation for the PDF of a random variable 
considered. Such a PDF equation determines the evolution of a stochastic process: 
according to Sect. 4.2.3 we can represent the PDF at every time in terms of 
random numbers (the correlations of a stochastic process are also determined by 
the PDF equation). The availability of a PDF evolution equation appears to be 
very helpful because of several reasons: 

• The PDF evolution equation determines an evolution of the stochastic process 
considered. Therefore, we can use the PDF evolution equation to validate and 
possibly to generalize the concepts used in Chap. 6 for the description of the 
evolution of a stochastic process. 

• The numerical PDF calculation is always affected by the number of realizations 
applied and the numerical method used for calculating the filtered PDF. On the 
other hand, the calculation of a PDF via its evolution equation can provide a 
PDF that is unaffected by the number of samples (and the filter width) applied. 

• The existence of a PDF evolution equation offers the chance to find analytically 
the PDF, which would imply a significant reduction of the problem considered 
(such that there is possibly only the need to determine the mean and variance of 
random variables). 
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• The existence of a PDF evolution equation offers the chance to find evolution 
equations for the mean, variance, and other moments of the process considered. 
Such moment equations can be solvable such that the temporal development of 
moments can be determined. 

Questions Considered. To take advantage of a PDF equation, we will derive in 
this chapter a PDF equation and analyze the relationship between equations for a 
stochastic process and for its PDF. In particular, we will consider the following 
questions: 

• Which equation describes the evolution of the PDF of a stochastic process? 

• How can we solve this PDF equation? 

• Which stochastic process equation corresponds to this PDF equation? 

• How can we determine stochastic equations for the modeling of any case? 

The derivation and solution of equations for moments (means and variances) of a 
PDF will be addressed, too. The latter questions will be discussed on the basis of 
differential equations for the evolution of both the PDF and the stochastic process 
considered, which simplifies the explanations significantly. In this chapter we 
focus on equations for one random variable, such that the characteristic features 
can be explained in a relatively simple way. The extension of the single-variable 
concepts to the case of several variables will be presented in Chap. 10. 



8.2 PDF Evolution Equations 

Let us address first the question of how the evolution of a PDF in time can be 
described. First, we will determine the general structure of an equation for the 
PDF of any stochastic process. Second, we will ask under which conditions this 
general equation can be simplified to an equation that is useful for the modeling of 
stochastic processes. 



8.2.1 The Kramers-Moyal Equation 

PDF Definition. The PDF of a random variable X is defined by the expression 
f{x) = <d{x-X)>. Here, S(x-X) refers to a delta function (see the explanations in 
Sect. 4.2.2). The expression y(x) = <d{x - X)> also can be used for a stochastic 
process that changes in time. The VDV f{x, t) at the time t is then defined by 

f{x,t)^{§{x-X{t)). (8.1) 
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At the later time t + M, the PDF is given by 

f{x,t + M) = i5{x-X{t + M)). (8.2) 

There is no need to make any assumption about the time interval ht here: it can be 
short or not. In order to calculate the evolution of/(x, t) in time, we have to relate 
the PDFy(x, t + M) at the later time to the PDF/(x, t) at the previous time. This is 
the question that will be considered in the following. 

Kramers-Moyal Equation. The best way to address the latter question is to 
consider first the instantaneous PDF involved in Eq. (8.2), 

5{x-X{t + M)=5{z). (8.3) 

For simplicity, the abbreviation z = x - X(t + M) is introduced here. To relate the 
instantaneous PDF at ? + A? to the instantaneous PDF at t, we consider the Taylor 
expansion of S (z) at Zq = x - X{t). This expansion reads 

^(z)=Z^^(z-Zo)", (8.4) 

where <5'"' refers to the w^-order derivative of (J(zo). The latter expression also can 
be written 



5{z) = 5{z,)+±\^ 
,1 = K ax 



(-1)" 
^^(z„-z)"<J(z„) 



(8.5) 



This expression results from the explicit consideration of the contribution at n = 0, 
the n^-order derivative of (5(zo) is written in terms of the corresponding derivative 
by X, and the rewriting (z - Zq)" = (-1)" (zq - z)" is used. The derivatives apply to 
all the bracket terms, but the only term that depends on x is S{z^ because z^^- z 
= X{t + /S.i) - X{i) is independent of x. By averaging Eq. (8.5) we obtain 

fix, t + At) = fix, + Z - — ^ , -, (8.6) 

n = iy ax ) n\ 

where the definitions oifix, t + M) and fix, i) are used. We write the term/(x, t) on 
the left-hand side, divide both sides by At, and take the limit A? — > 0, 



A'^o A^ „ = \\ dx) A(^o „!A? 

The left-hand side represents the partial derivative of/by t. Hence, we have 



dt „ = il dx 
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where we used the abbreviation 



z>'-'(..,). i. J'';7'7'^;r (8.9, 

A'^o n\Atf(x,t) 

The derivatives by x are written now as partial derivatives. The latter derivatives 
apply to the product D^"\x, t)J{x, t). Equation (8.8) is referred to as the Kramers- 
Moyal equation (Kramers 1940, Moyal 1949), and D^"\x, t) denote the Kramers- 
Moyal coefficients. 

Kramers-Moyal Coefficients. The Kramers-Moyal coefficients Z)*"'(x, t) can 
be rewritten by using the definitions Zf,- z= X{t + M) - X{t) and Zq=x- X{t), 

l{X(t + M)-X{t)Y5{x-X(t))) 
D'"\x,t) = hm \ VT^r-- '-■ (8.10) 

The properties of delta functions imply that S[x -X{t)) is zero except atX{t) =x. A 
convenient way to account for the condition X{t) = x is to use the definition of a 
conditional mean (see the corresponding explanations in Sect. 10.2.2) as an abbre- 
viation. Such a conditional mean is defined for any function g(X(t)) by 

-^{g{X{t)) Six - Xit))) = {g{Xit)) I Xit) = x) = {g(Xit)) \x,t). (8.11) 

The last expression represents a convenient rewriting of the previous expression: 
it refers to the condition that X ^ x at t. In terms of this definition, the Kramers- 
Moyal coefficients can be written 

{{X(t + At)-Xit)y\x,t) 

&"\x,t) = lim -^ '-. (8.12) 

A'^o n\At 

Marliov Process. The difference AX^X{t + At) - X{t) in the definidon of Z)*"' 
may depend on all the values of the random variable X at earlier times, this means 
on X(t - k At) with k^O, 1 , .... Depending on an appropriate choice of the random 
variable considered, it is relatively often the case that the influence of such mem- 
ory effects is relatively small, this means that AX can be considered to be deter- 
mined by the state X{t). Stochastic processes for which AX does only depend on 
the previous state X{t) are referred to as Markov processes (Gardiner 1983, Risken 
1984). Such Markov processes will be considered from now, which simplifies the 
application of the Kramers-Moyal equation significantly. For this case, the coeffi- 
cients ^''"'(x, t) are only functions of x and t. The Kramers-Moyal equation (8.8) 
then represents a differential equation of first-order with respect to time t. Com- 
bined with appropriate boundary conditions and the specification of an initial PDF 
y(x, to), Eq. (8.8) uniquely determines the PDFJ[x, t). 
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8.2.2 The Pawula Theorem 

Pawula's Theorem. The Kramers-Moyal equation (8.8) involves an infinite 
number of coefficients. Regarding the use of this equation it is, therefore, relevant 
to know how many coefficients should be taken into account. A relevant require- 
ment for dealing with the latter question arises from Pawula's theorem, which will 
be presented next. This theorem can be obtained like Schwarz's inequality (see the 
discussion of the correlation coefficient in Sect. 2.3.1). We introduce a non- 
negative function H(p) > as 

H{p) = l(^x' + p^x'^"•J\x,t\ 

\ / (8.13) 

= l^X^'' I x,t) + 2p(AX^''*'" I x,t) + p^AX^''*^"' I x,t). 



Here, AX = X(t + At) - X(t), and we assume that k> I and m > 0. The first two 
derivatives of H(p) by p are given by 

— = 2(aA''*+"' I x,t) + Ipl^AX^"*^"' I X,?), (8.14a) 

j2 rr 

^-^ = 2(az'*^''"|x,?). (8.14b) 

These two derivatives show that H{p) has a minimum at 

AX"-''^'"\x,t 



The minimum //„;„ of H(p) is given by 

, (ax'"^'" \ x,ty 

"'" ^ ' ^ 'aX''*""\x/ 

The function H{p) > for all p, this means we do also have //,^in > 0. Therefore, 
Eq. (8.16) implies that 

AX-' I xJ^I^AX^"^^"' \x,t)> (aX^"*"' I x,tf. (8.17) 

This condition is not useful for m = 0. Thus, we can consider m> \ from now. 
By dividing Eq. (8.17) by {Atf, taking the hmit At -> 0, and using Eq. (8.12) of 
the Kramers-Moyal coefficients D'-"\x, t), we find that 

(2/t)!Z)"*>(2/t + 2/w)!Z)''*^""> > [(2/t + m)!Z)<'*""]'. (8.18) 
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For simplicity, the dependence of Z)'"' on x and t is not indicated. Equations (8.18) 
represent the Theorem of Pawula (1967). 

Consequences of Pawula's Theorem. The consequences of Pawula's theorem 
can be seen by considering the cases that Z)'^*' = and Z)'"*^^'"' = 0, respectively. 
The right-hand side of Eq. (8. 18) is positive, and it has to be smaller or equal than 
zero for Z)'^*' = and Z)<2*+2".) ^ q. The latter constraints imply that 

!)<'*' =0 -^ !)<"+'"> =0, (8.19a) 

These two conditions can be simplified to one condition for any n>\, 

D(2") ^0 -^ Z)*'' = Z)''> = Z)"' = ■ ■ -Z)*"' = 0. (8.20) 

The coefficients Z)*'' and Z)'^' are unaffected by this constraint because 2 k + m 
cannot be smaller than 3 for A: > 1 and m> \. Thus, we do only have two options 
depending on whether or not the consequence of Eq. (8.20) does apply. 

a) All even coefficients Z)'""' for n > 1 are nonzero. The consequence of Eq. (8.20) 
does not apply then. We have to deal with an infinite number of coefficients. 

b) Any even coefficient Z)'^"* is zero, where n> \. The consequence of Eq. (8.20) 
does apply then. This means that we have Z)*^' = Z)*'" = Z)'^' = ... = £)<"' = Q. 

What is the consequence if a Kramers-Moyal equation is used that is not consist- 
ent with Pawula's theorem (e.g., an equation that involves nonzero coefficients up 
to fourth order)? Pawula's theorem corresponds to the consideration of the quad- 
ratic moment H(p) as a non-negative number. Thus, we may find negative squared 
moments if Pawula's theorem is not satisfied. Moments represent integrals over 
the corresponding PDF. Hence, the PDF must have negative values if Pawula's 
theorem is disregarded. 



8.2.3 The Fokker-Planck Equation 

Fokker-Planck Equation. The consideration of an infinite number of even 
Kramers-Moyal coefficients implies the problem of providing all the coefficients 
as functions of x. Apart from that, an equation with an infinite number of terms is 
difficult to treat numerically. Hence, we will apply the option b) described in the 
preceding paragraph, i.e., we neglect coefficients Z)'"' with « > 3. Hence, we will 
consider from now on the equation 

df{x,t) _ dD''\x,t)f(x,t) ^ d'D''\x,t)f{x,t) ^g2i) 

8t dx dx^ 

This equation represents the Fokker-Planck equation (Fokker 1914, Planck 1917). 
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According to Eq. (8.12), Z)'" and£)'^> are defined by 

l^x\xA) 



D'"(x,t)- 



lim 

A(->0 



At 



D'''(x,t)- 



lim 

At-fO 



AX'\x,t 
2 At 



(8.22) 



Under which condition is the neglect of Z)*"' with « > 3 justified? The answer to 
this question is closely related to the consideration of the continuity of the sample 
path of stochastic processes. By considering an infinitesimal time increment At it 
is relatively often the case that the change AX^ X{t + At) - X{t) is bounded (i.e., 
relatively small). Such stochastic processes have a continuous sample path. The 
consideration of such a process implies the neglect of Z)*"' with « > 3 (see Gardiner 
1983). In other words, jump processes (instantaneous changes AAf that may be 
very large) with a discontinuous sample path are taken into account if coefficients 
Z)(") with « > 3 are involved. 

Mean Equation. The implications of the Fokker-Planck equation (8.21) for the 
evolution of moments of the PDF /(x, t) will be considered next. A main purpose 
of this discussion is to provide insight into the relevance of the model parameters 
of the Fokker-Planck equation. By multiplying the Fokker-Planck equation (8.21) 
by X and integration over the sample space we obtain 



I- 



df{x,t) 



dt 



dx ■■ 



r dD^'\x,t)f{x,t) ^^ ^ , d'-D^'\x,t)f{x,t) 
' dx ' dx^ 



dx. 



(8.23) 



This equation has to be rewritten in terms of moments ofj{x, t), i.e., as functions 
of X multiplied by the PDF/(x, t) and integrated over the sample space. To find the 
corresponding expressions we write 

^fx/(x,0^x^-J ^^^"'^^"'^)-^^^"'^^ A- + J^Z)"'(x,0/(x,0^x 
dt dx dx 



+! 



dx 



_d&'\x,t)f{x,t) 



dx 



^^.d^dD-^MlMdx 



dx 



dx 



-(xZ)('>(x,0/(x,0)l + \D''\x,t)f{x,t)dx 



(8.24) 



.5Z)"'(x,0 



dx 



f{x,t)+x&^\x,t) 



df{x,t) 



dx 



■(Z)<^>(x,0/(x,0)l. 



The integral on the left-hand side is equal to <X>, and the second integral on the 
right-hand side represents <£)*''>. The other integrals on the right-hand side do 
vanish because the PDF f(x, t) and its derivatives are zero at infinity. Therefore, 
Eq. (8.24) can be written 



dt 



D' 



(8.25) 
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The partial derivative by t can be replaced here by the normal derivative because 
<X> and <Z)*'*> are only functions oft. Hence, <£)'''> determines the transport of 
the means <X>. For that reason, Z)*'* is called the drift coefficient. 

Variance Equation. The variance equation can be obtained similarly. We mul- 
tiply the Fokker-Planck equation (8.21) by x^ and integrate over the sample space, 

^ dt ^ dx •" 5x' 

In correspondence to the rewriting of Eq. (8.23) we write 

2 



-\x'f{x,t) dx = _r^^L^^!!MZM dx + \^DHx,t)f{x,t) dx 



' dx 



^, d&^\x,t)f{x,t) 
dx 



^^_^d^d_D^%^M^dx. 



(8.27) 



dx dx 



The integral on the left-hand side is equal to <X->. The first and third term on the 
right-hand side disappear as the corresponding terms in Eq. (8.24). Thus, 



d(X' 



= 2\x&»{x,t)f{x,t) dx - 2f^ g^'"(^^0/(^^0 ^^ 

dx 

^ 2(xZ)<") - 2|^f^^!!MZM dx + 2J^Z)'^'(x,0/(x,0 dx. 



dx dx 

The second term of the last line is equal to zero because we take an integral over a 
derivative. By rewriting the last integral we obtain 

^ '- = 2{XD^'^^ + 2{p^^'^Y (8.29) 

Here, the partial derivative by t is replaced by the normal derivative by t. Instead 
of considering the equations for second-order moments, it is more convenient to 
consider equations for the variance 

{x^) = [{x-{x)Y) = {x^-)-{x)\ (8.30) 

By differentiating this variance expression by t we obtain 

dlr^ dix') dixf d{x') ^ ^d(x) 

-^ '- = ^ '-- ^ ' ^-^ L-2lx)-^^. (8.31) 

dt dt dt dt ^ ' dt 

The use of Eqs. (8.29) and (8.25) then implies the following variance equation. 



dlX^ 



dt 



2(X Z)'" + 2(Z)"' - 2lX)lD''' = 2{XD'» + 2 Z)'^' ) . (8.32) 
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In the last expression we used <XZ)"' > = <ZZ)"*> - <X> <Z)*">, which may be 
proven by distributing the left-hand side. Hence, variance is produced by <Z)*^'> if 
Z)*^* is positive. The corresponding increase of the width of the PDF characterizes 
a diffusion process, which is the reason why Z)'^' is called a diffusion coefficient. 
An equilibrium state may be reached asymptotically if the first term on the right- 
hand side of Eq. (8.32) appears with a negative sign, i.e., if this terms describes a 
dissipation (or destruction) of variance. 

Correlations. The correlation between X{t) and X{t) can be calculated on the 
basis of the Fokker-Planck equation (8.21). Without loss of generality we assume 
that t<t = t + r, where r is any non-negative time. As shown in exercise 8.3. 1, the 
correlation between X(^) anAX{t + r) is found to be determined by the equation 



d{X(t)X{t + r)) ^ 

-^ ^ = (X(OZ)'"(X(? + r),? + r)). (8.33) 

The correlation at r = is determined by the variance equation (8.32). We see that 
the correlation is unaffected by the diffusion coefficient Z)'^'. Thus, correlations 
are not produced, but correlations relax according to the model provided by Z)'". 



8,3 Solution to the Fokker-Planck Equation 

Let us illustrate basic characteristics of solutions to the Fokker-Planck equation 
(8.21) by considering the following example, which enables the calculation of an 
analytical solution. The equation considered is given by 

%^ = -A[;.(O.G(0(x-(x))]/(x,0.^^^^^#^. (8.34) 

at ox ox 

The drift coefficient Z)*'' is a linear function of x. The inclusion of <X> in Z)*'' 
represents a convenient writing: the term G <X> could be also combined with the 
drift term F. Such a linear model for Z)*'' is well appropriate to characterize near- 
equilibrium processes. The diffusion coefficient D{t) is assumed to depend only 
on t, which is often a convenient assumption. Equation (8.34) will be combined 
with natural boundary conditions, this means we have/(x, ?) — > for |x| — > oo. 



8.3.1 The Solution Approach 

Solution Approach. The solutions /(x, i) to the Fokker-Planck equation (8.34) 
involve (/) information about the initial PDFy(X(,, ?„), and (//) information about the 
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transition from the initial PDF to any asymptotic PDF, which is determined by the 
PDF evolution equation. It is very helpful to represent the solution such that these 
two ingredients are separated from each other: this simplifies significantly the use 
of the solution for any initial PDFs. The latter can be achieved in the following 
way. First, we represent the PDFy(x) as 

fix,t) = \fix,t;xQ,tf^)dXf,. (8.35) 

liere,J{x, t, Xq, t^) represents the two-point PDF, which is defined by 

f{x,t,x„t,) = {S{x-X{t))Six,-X(t,))). (8.36) 

The consistency of Eq. (8.35) can be seen by using the definition (8.36) in Eq. 
(8.35) and performing the integration. This recovers the definition (8.1) of^^, t). 
Second, we introduce the conditional PDFy(^, t \ x^, Iq) by the relation 

f(,tl, t )- /fc^'^o'^o) - {Si^-X{t))S{x,-X{t,))) 

f{^„h) {S{x,-X{t,))) (8.37) 

= {5{x-X{t))\X{t,) = x,) = {5{x-X{t))\x„t,). 

The last expression of the first hne results from the use of the definitions of the 
two-point VDV f{x, t, x^, t^ and initial PDF/(xo, ?o)- The expressions in the last line 
make use of the definition (8.11) of conditional means. Hence, /(x, t \ Xq, t^ is the 
PDF ofX{i) under the condition that X(?q) = Xq (see also the discussions related to 
conditional PDFs in Sect. 10.2.2). By relating the two-point PDF to the condition- 
al PDF according to Eq. (8.37) we can write Eq. (8.35) as 

f{x,t) = \f{x,t\x^,t^) /(xo,?„) dx^. (8.38) 

In this way, the PDF /(x, t) is calculated as an integral over the conditional PDF 
/(x, t I Xq, ?o) multiplied by the probability for having the initial value Xq. The 
advantage of this approach is that we can calculate a general expression for the 
conditional PDF/(x, t \ Xq, ?o), which is independent of the initial PDF/(xo, t^). The 
PDF/(x, t) can be obtained then for any initial PDF/(xo, t^ by integration accord- 
ing to Eq. (8.38). 

Conditional PDF Equation. How is it possible to find the conditional PDF 
y(x, 1 1 Xq, ?o)? Using Eq. (8.38), the Fokker-Planck equation (8.34) can be written 



8/(x,?|xo,?o) d[F{t) + G{t)[x-{X)^f{x,t\x^,t,) d^D{t)f{x,t\x„t,) ] 
dt ^ ' - . : 

■ fixoJo) dxo =0. 



^ \ dt 8x dx^ 



(8.39) 
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Hence, the conditional PDF/(x, t \ Xq, ?o) has to satisfy the Fokker-Planck equation 
(8.34), too, this means the conditional VT>¥ j(x, t \ Xq, ?o) has to satisfy the equation 

dfix,t\x„t,) _ d[Fit) + G(t){x-{x))]f{x,t\x„t,) d^D{t)f{x,t\x„t,) 
dt dx dx^ 

(8.40) 

Conditional PDF Initial Condition. The initial condition that is required for 

the calculation of the conditional PDFy(x, t \ Xq, tg) can be derived from Eq. (8.37), 

{Six-X(t,))Six,-Xit,))) _ {Six-x,)S(x,-Xit,))) 
"' {S(x,-X(t,))) {S(x,-X(t,))) (8.41) 

= S(x-Xo). 

Here, we used the definition of/(x, t \ Xq, Iq) att^tg and the sifting property of the 
delta function to obtain the last expression. 



8.3.2 The Solution to the Fokker-Planck Equation 

Conditional PDF Model. The asymptotic conditional PDF fix, t \ Xq, t^ that is 
implied by the Fokker-Planck equation (8.40) is a normal PDF: see exercise 8.3.2. 
In addition, a normal PDF can satisfy the delta function initial condition (8.41) for 
the conditional PDF (see exercise 8.3.3). Therefore, it makes sense to ask whether 
the conditional PDF can be described in each instance by a normal PDF. To prove 
the suitability of this idea, we model the conditional PDFy(x, t \ Xq, t^ by a normal 
PDF, 

f{x,t\x„t,) = ^^QX^\- ^''~"' \. (8.42) 

Here, a(t) represents the mean value and P{t) is the variance of this conditional 
PDF model. To be consistent with the initial condition (8.41), a and /? need to 
have the initial values a{t^ = Xq and /i(?o) = (see exercise 8.3.3). Another view of 
looking at the assumption (8.42) is the following: we ask under which conditions 
we may have a normal PDF as solution of the Fokker-Planck equation. By using 
expression (8.42), the solution of the Fokker-Planck equation (8.34) is given by 

/(x,0 = j^=i=exp|-^^^l/(xo,^o)«'xo. (8.43) 

^2nP [ 2// J 

Here,y(X|), t^ can be any initial PDF. It is worth noting that a depends on Xq via its 
initial value cdto) ^ Xq. 
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Conditional PDF Model Validity. To prove the suitability of the assumption 
(8.42) we have to show that Eq. (8.42) can satisfy the Fokker-Planck equation 
(8.40). To address this question we write Eq. (8.40) as 



^"^^^'^l;'"^°^^^^-G(/)-[7^(0 + G(0(x-(X)f-^^"'^l^°'^»^^^" 

J yx, 7 Xq , f Q J J \X, I Xq , 7q J 



+ D{t) 



d'^f{x,t\x^,t^)ldx^ 



(8.44) 



/ (x,t I XQ,rQ) 
The derivatives required in this relation follow from Eq. (8.42), 



df{x,t\x^,t^)ldt ^' (x-«)' , x-a 



J \X,t I XqjJqJ 

df{x,t\xa,t^,)ldx 

J (X, f XQ,fQj 



2/3 2/?' 



/?' + ■ 



-a' 



x-a 



d^f{x,t\x^,t^)ldx'^ _{x-af 1 
f{x,t\x^,t,) /?' /?■ 



(8.45a) 
(8.45b) 
(8.45c) 



Here, a' and /?' refer to the derivatives of a and /?by t, respectively. By making 
use of these derivatives in Eq. (8.44) we find the relation 

P' {x-af 



ip ip^ 



n, ^' - « , 



= -G(t) + [7^(0 + G(0 (x - « + « - (X))] ^-^ + D{t) 



{x-af 



7 



(8.46) 



Here, - « + « was added to the second term on the right-hand side to simplify the 
following derivations. By considering terms that are independent of x - a, linear 
in X - «, and quadratic in x - «, the latter condition can be written 

= P^-^--^ L5'-2/?G(0-2Z)(0] + ^[«'-G(0(«-(x))-7^(0]- 

L p L p j p 

(8.47) 
The terms that are independent of x - « and quadratic are combined here because 
they have the same bracket term. This condition has to be satisfied for all x. The 
resulting requirement is that the two bracket terms have to disappear. We obtain 
two equations for a and fim this way, 

da 
dt 

dp_ 
dt 



-G{t)[a-{X))+F{t), 
2G{t)/3 + 2D{t). 



(8.48a) 
(8.48b) 
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The fact that it is possible to satisfy the condition (8.47) for all x means that the 
model (8.42) represents an appropriate model for the conditional PDF, i.e., this 
model satisfies the Fokker-Planck equation (8.40) for the conditional PDF. 



8.3.3 Process and Conditional PDF Statistics 

Relations Between Process and Conditional PDF Statistics. It is interesting 
to compare the Eqs. (8.48) for the mean a and variance /?of the conditional PDF 
model (8.42) with the equations for the mean <X> and the variance <X' >, which 
follow from the Fokker-Planck equation (8.34). The simplest way to obtain the 
latter equations is to use the general Eqs. (8.25) and (8.32) for the mean <X> and 
variance <X^> for the case considered, 

-LA = ^z)("^ ^ (^j7(^t) + G{t) x'j = F(t). (8.49a) 

3li . 2(15<'>) + 2(Z)<^') = 2{x(Fit) + Git) X)) + 2Dit) ^^^^^^ 

= 2G(0(i^') + 2Z)(0. 

Equations (8.49) reveal a significant similarity to Eqs. (8.48) for the model param- 
eters a and /? of the conditional PDF. To see the difference in more detail, let us 
use Eqs. (8.48) and (8.49) to find the differences a - <X> and /?- < X^- >, 

d 



dt 

d_ 
dt 



[a -{X))= G{t) [a - {X)), (8.50a) 

(/]-{x'))=2G{t)(fi-{x')). (8.50b) 



The solutions of these equations, which satisfy the initial conditions «(?o) ^ ^o ^^d 
yfl(?o) = 0, are given by 

a = {X) + {x,-{X,))exp\\Gis)ds\, (8.51a) 



^ = i^X^j- i^xA e\p\2]G(s)dsy (8.51b) 

For a positive G, these solutions imply model parameters a and /? that approach 
infinity asymptotically - which does not make sense. Consequently, G has to be 
negative. 
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Asymptotic Evolution. For a negative G{t), a and /? relax asymptotically (i.e., 
for ? — > oo) to the means and variances of/(x, t), 

a = {x), ^ = {^')- (^-^2) 

For this case we have a and /?that are independent of Xq. Equation (8.43) for the 
PDFX^, t) reduces for this case to 



271 X') ^[^ J2n(X 



2 X^) h^ yA 2(X 



(8.53) 
where the normalization condition for/(xo, ?o) is applied. Hence, independent of 
the initial PDF the PDFy(x, i) does relax asymptotically to a normal PDF. 



8.4 Stochastic Differential Equations 

One way of modeling the evolution of stochastic processes was considered in 
the previous two sections where equations for the PDF of random variables were 
introduced. It was argued that the reduction of the Kramers-Moyal equation (8.8) 
to the Fokker-Planck equation (8.21) represents (at least under many conditions) 
the most suitable way of constructing a PDF transport equation. An alternative is 
to postulate differential equations for the evolution of stochastic processes. These 
differential equations determine all the coefficients in the Kramers-Moyal equa- 
tion. Therefore, this approach does result, too, in a PDF transport equation. The 
relations between these two approaches will be considered in this section. 



8.4.1 Nonlinear Markovian Stochastic Equations 

Approach. The structure of appropriate stochastic differential equations will be 
determined in the following way: 

• First, the stochastic difference equations developed in Chap. 6 will be used to 
determine the general structure of stochastic difference equations. 

• Second, the general stochastic difference equation obtained in the first step will 
be represented as a stochastic differential equation. 

• Third, the stochastic integration will be defined, which is required for the calcu- 
lation of solutions of stochastic differential equations (and the reproduction of 
the stochastic differential equation considered, see below). 
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Stochastic Difference Equation. The first step requires the generalization of 
the stochastic difference equation models that have been developed in Chap. 6. All 
the models considered can be covered by the equation 

X — X \w 

" ■'-^ ^a{X„_„t„_,) + b{X„_„t„_,)—^. (8.54) 

Ar A^ 

Here, X„ represents the variable considered (the particle position, particle velocity, 
or population density), and we have Aff^,_, = (A?)"^ £„-i- The random variable £„_i 
is normality distributed and characterized by <£,> = and <£}, £„> = S,,,,,. We will 
assume that s„_i is independent of Xq. The coefficients a(X„_i, ?„_[) and b{X„_i, ?„_[) 
are any functions of X„_, and ^„_] (the models considered in Chap. 6 did not involve 
an explicit time dependence, but such a t„_i dependence may be relevant to other 
problems). No assumption about the time interval A? is made here. Regarding the 
following explanations it is worth noting that X„_i is independent of e„_i because 
X„_i is only affected by previous values £„_2, £„-}, .... Hence, AW„_i is independent 
of the coefficients a(X„_y t„_i) and b{X„_y ?„_,). 

Stochastic Differential Equation. The second step is the transition from the 
stochastic difference equation (8.54) to a differential equation. By considering an 
infinitesimal time interval A? — > and defining time t by t = n At, the stochastic 
model (8.54) can be written as 

^{t) = a{X,t) + b{X,t)^{t). (8.55) 

The change of the stochastic process X is fully determined by a(X(t), t), b(X(t), t), 
and dWldt{t). Therefore, we find that Eq. (8.55) describes the evolution of the 
stochastic process X as a Markov process: the future properties of X are fully 
determined by the present state at t. Let us have a closer look at the properties of 
the derivative dW I dt of a Wiener process. The limit A? — > does not change the 
statistical properties, i.e., dWI dt is normally distributed as AW„_il At. The mean of 
dW/ dt can he derived by means of the meanof AfF„_[/A?, 

Correspondingly, the mean oidWI dt is given by 

I dW \ 
y-(OHO. (8.57) 

dt I 

The properties of correlations of dW/dt are determined by the relation 

'aw aw \ 
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For a continuous time t = n M and ?' = m A?, Eq. (8.58) can be written 

dW dW \ I \ 

^—{t)^—{r))dt^de{t-t\ (8.59) 

dt dt I 

The differential dO of the theta function is zero for negative or positive arguments 
t- t but not for ^ - ?' = 0: at this value the theta function jumps from zero to one 
such that dO^ 1. Thus, Eq. (8.59) agrees with the consequences of Eq. (8.58). In 
terms of the definition of the delta function we can write 

dW dW \ 

^^{t)^—{t')) = 5{t-t'). (8.60) 

dt dt I 

Stochastic Integration. The third step is to define the stochastic integration, 
such that solutions of the stochastic differential equation (8.55) can be calculated. 
This question will be addressed by asking under which conditions the integration 
recovers the stochastic differential equation (8.55) considered. For doing this we 
integrate Eq. (8.55) from tiot + dt, 

X{t + dt)-X{t)= \a{X{s),s)ds+ \ b{X{s),s)—-{s)ds, (8.61) 

where dt ^ Q h an infinitesimal time interval. The integration can be defined in 
several ways. One way, which is called the Ito definition, is to take the function 
values oia{X{s), s) and b{X{s), s) at X{t) and t, such that 

X{t + dt) - X{t) = a{X(t), t) dt + b{X(t), t) dW(t). (8.62) 

Here, dW(t) is given by dW{t) = W{t + dt) - W(t). Another way of defining the 
integration, which is the Stratonovich definition, is to take a{X{s), s) and b{X(s), s) 
at the mean value [X{t + dt) + X{t)] 1 2 and t. For non-random variables both defi- 
nitions of the integration provide the same result for continuous fiinctions. How- 
ever, this is not the case for random variables because dW scales with (A^)"^: the 
Stratonovich definition results in a stochastic differential equation that involves a 
deterministic drift term in addition to the term a{X{t), t) (see Gardiner 1983 and 
Risken 1984). In the following we will apply the Ito definition of the stochastic 
integration because the implied Eq. (8.62) recovers the structure of Eq. (8.55). 



8.4.2 Relationship to the Fokker-Planck Equation 

Next, let us compare the implications of the stochastic model (8.55) with the 
consequences of the Fokker-Planck equation (8.21). The comparison of stochastic 
processes defined in different ways requires, first, the comparison of one -point 
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statistics, and, second, the comparison of correlations. Therefore, we will calculate 
the evolution of the PDF and correlations implied by the stochastic model (8.55) 
and compare these results with the implications of the Fokker-Planck equation. 

PDF Equation. To find the PDF equation that is implied by the stochastic 
model (8.55) we consider the Kramers-Moyal equation (8.8), 



df(x,t) _^( 5 I „„ 



Z\-^\D'"\x,t)f(x,t). (8.63) 

ot „ = i\^ dxj 

The Kramers-Moyal coefficients are defined by Eq. (8.12), 

{{X(t + At)-X(t)Y\X(t) = x) 
D'-"\x,t) = lim ^ L. (8.64) 

According to Eq. (8.62), the stochastic differential equation (8.55) provides for the 
change dJ({t) = X{t + A^ - X{t) the expression 

X{t + At) - X{t) = a{X(t), t)At + b{X(t), t) AW(t) . (8.65) 

In terms of this expression we find for the Kramers-Moyal coefficients 

([a{X{t),t)At + b{Xit),t)AW(t)]" I X(t) = x) 

&"\x,t) = lim ^ L 

A'->o n\At 

= lim -i — - — ( a(x,t)yfAt+b(x,t) — p=-^ 
A'^o n\At \L Va7 _ ^ 

The second line makes use ofX(t) = x. The consideration of the factor (A^)"'^ sim- 
plifies the calculations of Kramers-Moyal coefficients, for which we find 

&'\x,t) = lim ^^^(a(x,t)y[At + b(x,t)^^Ml\ = a(xA (8.67a) 

A'^" At \ ^ ' Va7 / 



D^^\x,t)= lim la-{x,t)At + 2a{x,t)b{x,t)AW{t) + b^{x,t) 



A'^o2A? \ At 

= -b\x,t). (8.67b) 

Z)">(x,0 = Z)'**'(x,0 = ■■■Z)<"'(x,0 = 0. (8.67c) 

The first two expressions result from the properties of AW{t) and the limit At — > 0. 
The validity of Eq. (8.67c) can be seen by having a closer look at Eq. (8.66). The 
bracket term provides for all n any finite value. The ratio {At)"^^ I At is always zero 
for a vanishing At'ifn > 3. Hence, all Kramers-Moyal coefficients have to vanish 
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except Z)*'' and D*^'. Thus, the PDF of the stochastic model (8.55) is determined by 
a Fokker-Planck equation with Z)''* = a and Z)'^' = b^ 1 2. Hence, by using Z)*'' = a 
and Z)'^' = b^ I 2 we can apply Eqs. (8.25) and (8.32) for the mean <X> and 
variance < X^> implied by the Fokker-Planck equation (8.21) to obtain the equa- 
tions for the mean and variance that follow from the stochastic model (8.55). 

Correlations. Next, we will use the stochastic model (8.55) for the calculations 
of correlations between X(?) and X{f). Consistent with the approach in Sect. 8.2.3 
we assume again that t < t = t + r, where r is any non-negative time. We may 
consider Eq. (8.55) att + r instead at t, 

'^^^' ^ ''' = a{X(t + r),t + r)+b{X{t + r\t + r) — {t + r). (8.68) 

The differentiation of Xby t + r can be replaced by a derivative by r. We multiply 
this equation with X(t) and average, 

(x(0^^4^) = (l(0«(X(r + r),r + r)) 

+ lx{t)b{X{t + r),t + r)^{t + ry). 

The last term disappears: dW I dt ai t + r is independent oiX{t) and X{t + r), and 
dWIdt vanishes in the mean, 

lx(t)b[X{t + r),t + r)^{t + r)\ = ^X{t)b{X(t + r),t + r))l^{t + r)\ = 0. 

(8.70) 
By writing the derivative by r in front of the bracket term we obtain, therefore. 



d{Xit)X{t + r)) ,^ 

Xit)a{X{t + r),t + r)). (8.71) 



dr 

We may replace X and a by the corresponding fluctuations because the mean of X 
and a do not change the correlations functions. Thus, we obtain the equation 



d(Xit)X(t + r)) 

-^ '- = {X(t)3{Xit + r),t + r)y (8.72) 

This equation recovers Eq. (8.33), which is a consequence of the Fokker-Planck 
equation (8.21) if Z)''' = a is taken into account. In combination with the obser- 
vation that both the stochastic model (8.55) and the Fokker-Planck equation (8.21) 
imply the same PDF evolution equation, we find that the stochastic model and the 
Fokker-Planck equation describe the same stochastic process (if the Z)"' = a and 
Z)P> = Z7V2apply). 
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8.4.3 Linear Markovian Stochastic Equations 

Let us consider an example to make the analysis of nonlinear stochastic models 
in the previous sections more explicit. We consider the following linear stochastic 
Markovian differential equation, 

at T T at 

which is relevant regarding the discussion in Sect. 8.5. This equation is written 
according to the Brownian motion velocity model (6.59b). Here, the constant D is 
a diffusion coefficient, and the constant r is the characteristic relaxation time scale 
of fluctuations. By averaging Eq. (8.73) the mean value <X> becomes 

d(x) 

^-^ = 0. (8.74) 

dt 

Hence, the mean <X> has to be a constant, this means <X> = <Xq>. 

PDF. The combination of the Fokker-Planck equation (8.21) with the relations 
Z)"> = fl = -(x - <X>) I T and Z)<^> = b^ 12= Dl {2 r^') between the coefficients of the 
PDF equation (8.21) and the stochastic model (8.55) shows that the evolution of 
X{t) is described by the Fokker-Planck equation 

df{x,t)_l d[:c-{X))f{x,t) ^ D d'f{x,t) 

dt T dx 2t^ dx^ ' 

This equation represents a specific case of the PDF equation (8.34), which can be 
solved analytically. In particular, we have here the case that 7*'= 0, G = -1 / r, and 
D in Eq. (8.34) corresponds to Z)/(2 t^) in Eq. (8.75). Hence, the solutiony(jc, t) of 
Eq. (8.75) is given by Eq. (8.43), 

^('^'^)"I^^=^'^^P ~ oT \f(''o,to)dx,. (8.76) 

p7ij3 [ 2/3 J 

Here,y(xo, t^ refers to any initial PDF. The model parameters a and y^ satisfy the 
Eqs. (8.48), 

'^-.-'^ziA. (8^77a, 

dt T 



dp___2p_ p 

dt TV 



+ —. (8.77b) 



The initial conditions for these equations are given by «(0) = Xq and /5(0) = 0. 
For simplicity, we assume here that tg = 0. The solutions of Eqs. (8.77) become 
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very simple by writing these equations in the following way, 



dt 
d 



(«-(x))=-i(«-(x)), 

D 



dt\ It 



/ 9 

T\ IT 



(8.78a) 
(8.78b) 



Correspondingly, the model parameters a and /? are given by the functions 

t 
= {X) + [x,-{X))e~^, (8.79a) 



a 



13 = e ^ = — 1-e 

2t It 2r 



2' A 



(8.79b) 



Asymptotically, a realizes <X> and P realizes Dl {It). 

Correlations. For the calculation of correlations ofXit is helpful to consider 
the formal solution of Eq. (8.73), which is given by 



Xit) = {X) + [X, - {X))e' + ^ je""' ^(.) ds. 

T a ds 



(8.80) 



By setting t = we can see that this solution recovers the initial condition Xq. 
The fact that X{t) satisfies the differential equation (8.73) can be seen by taking 
the derivative of this expression. Therefore, the fluctuations X{t) = X(t) - <X> of 
XaK given by 



X{t) = X^e ' +- — je ' [s)ds. 

T ds 

To calculate the correlation function we multiply X{t) with 



XiV) = X^e-^+^]e^'^{s')ds\ 
T ds' 

where f is any time. By taking the average of this product we find 

imxif)) 



(8.81) 



(8.82) 



X,e'-^.^\e'^'^{s)dXx,eU^\e'^'^{s')ds' 
T ds II T a ds' 



Xa )e 



t+t 7~\ / /"' ^~^ ' "■^ 



^Ue ^ 



^{,)^(,')\ds'ds. 
ds ds' 



(8.83) 
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In accordance with the fact that £•„_] is independent of Xq (see the discussion of 
Eq. (8.54)) we find that dW/ds and dWI ds' are uncorrelated to Jf,, , which leads to 
the last expression. The last term in Eq. (8.83) can be rewritten, 



t-s t'-s' 



—^\\e 'e ' ( \s) — -\s')\ ds' ds = —^\\e 'e ' S{s'-s)ds' ds 

Too \ ds ds I ^00 

D '/; -'-^^^ deis'-s) , , , D'r -'-^^^x^i , X ^/ x\, 

= —\\^ ' \, ' ds'ds = —\e ^ [0{t'-s)-9{-s)\ds 



ds' 

j-^ , i+r'-2s j-^ min(r,r') _^ 

= ^-}e ' 0[f-s)ds = —^ \ e 

2" ^0 



(8.84) 



ds 



t'2 



t+!'-2m\n(t,t') t+t' 

-e ' 



D_ 

2r 



e "" - e 



The first line applies the definition of the correlation of dW/ ds and dW/ds'. In the 
second line, the sifting property of the delta function is used, the delta function is 
replaced by the derivative of the theta function, and the integration over ds' is 
performed. Here, the term d{-s) is zero due to the variation of ^ considered. The 
remaining expressions provide then an explicit function. Hence, Eq. (8.83) reads 



X(t)Xit')) = ^X„)e ^ + 

D 

— ( 

It 



D_ 

It 

X- 



\<-t'\ t+f 
e ' -e ' 



2t] 



(8.85) 



Without loss of generality we set ^ < ^' = ? + r, where r is a non-negative time 
interval. 



X{t)X{t + r)) = ^e'^ + 



X, 



D_ 

2r 



D_ 

It 



Xr, 



D_ 

2t 



The latter result can be written more conveniently as 



where the variance is defined by 



X\t) 



D_ 

2r 



+ X 



D_ 

~2t 



e ". 



_2f> 
J 

(8.86) 
(8.87) 

(8.88) 



The last to expressions agree with the corresponding consequences of the Fokker- 
Planck equation (8.21) (see exercise 8.4.1). 
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8.5 Non-Markovian Stochastic Differential Equations 

In the previous sections we developed mathematical concepts for the modeling 
of the evolution of stochastic processes. We did not address at all the question of 
how these concepts can be applied to the solution of a problem, this means how a 
case considered can be modeled in terms of a stochastic differential equation. The 
stochastic differential equations introduced above are general with one exception: 
the essential assumption applied was the consideration of a Markovian stochastic 
process (the assumption that the future statistical properties of a stochastic process 
are fully determined by the present process properties). However, this assumption 
is usually not rigorously satisfied: most real processes represent non-Markovian 
processes. Let us consider how we can deal with this problem by considering a 
relevant but not too complicated problem: the motion of a molecule. 



8.5.1 Markovian and Non-Markovian Velocity Models 

Velocity Model. Let us consider the following Markovian linear stochastic 
model for the velocity v of one molecule in one direction (Heinz 2003, 2004), 

^(0 = -^^^ + Jf 5". (8.89) 

dt T \iT dt 

Here, <;;> is the mean molecular velocity, r is the characteristic relaxation time 
scale of velocities, and e refers to the specific kinetic energy of molecules, which 
is related to the equilibrium variance <v^ > of the velocity by e = 3 <i}^ > / 2. 
By averaging Eq. (8.89) we see that <i;> has to be constant. For simplicity, the 
parameters r and e are considered to be constant. This velocity model represents 
the continuous version of the discrete velocity model (6.59b) for the motion of a 
Brownian particle, where the diffusion coefficient D is replaced byZ)=4er/3 
(the only difference is given by the consideration of a nonzero mean velocity <u> 
here). 

Acceleration Model. The velocity model (8.89) represents a reasonable model, 
but its simplicity implies some shortcomings (see the discussion in Sect. 8.5.4). 
A model that does better agree with the motion of molecules in reality is given by 
the following stochastic equation for the acceleration dvl dt (Heinz 2003, 2004), 



(8.90) 




d dv , , 


1 


dtdt''^- 




^f 
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The velocity model (8.89) can be recovered by multiplying this equation by r^ and 
taking the limit r^ — > 0. The bracket term is equal to zero in this case, which corre- 
sponds to the velocity model (8.89). The idea of Eq. (8.90) is that there are non- 
zero deviations from the velocity model assumption (8.89), which are given by the 
bracket term. The latter deviations lead to accelerations that relax to zero with a 
characteristic time scale z}. For simplicity, r^ is considered to be constant as <v>, 
T, and e. It is interesting that the model (8.90) also can be written as a system of 
equations by considering the set of variables (v, a), where a = dv I dth the particle 
acceleration. According to the definition a = dv I dt and Eq. (8.90), the equations 
for these variables are given by 



dv 
dt 



{t) = a. 



da , , 


1 


^'^> = 




dt 


^f 



-a{t)- 



v(t)-{v) 




(8.91a) 
(8.91b) 



These equations represent a linear Markovian stochastic equation system for the 
process (v, a), this means we have a Markovian acceleration model. The fact that 
<i;> is constant implies that <a> = 0. 

Velocity Model Implied by Acceleration Model. Let us write the acceleration 
model (8.90) as a velocity model to compare it with the velocity model (8.89). 
Equation (8.90) can be seen as a nonhomogeneous linear first-order equation for 
dv/ dt. The formal solution of this equation reads 



dv 
dt 



-tlzf 1 r -(t-s)lrf 

■ a^e H J e 




ds . 



(8.92) 



The validity of this writing may be seen by proving that this solution provides the 
initial acceleration a(0) = Qq, and it satisfies the differential equation (8.90). On 
the right-hand side we have terms of different nature: the term with (v(s)- <v>)l r 
is a systematic contribution, and the terms that involve Aq ^nd dWI ds are random 
terms. To simplify the writing we combine the last two terms to a stochastic force 



f{t) = a^e 




t-s)ii, dW 



' -^{s) ds. 
ds 



(8.93) 



This force vanishes in the mean, <J{t)> = 0, and its initial value is^^ ^/(O) = Oq- 
The simplest way to see the statistical properties oiJ{t) is to use Eq. (8.93) for the 
derivation of the following equation for J[t), 



4i, 

dt 



1 



f- 




(8.94) 
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which enables the use of the analysis results of the linear stochastic model (8.73). 
By replacing D and v in Eq. (8.73) by Z) = 4 e / (3 r) and Vf, respectively, we can 
use the correlation (8.85) for finding the following force correlation, 



{mfin)=^e '' + if;-) 



2e 



^TTf J 



e ' . (8.95) 



dv 


-{'.' 


= 


dt 



The setting of the initial force variance is up to us: we set <fo^> = 2 e/ [3 r z)] such 
that the last term in the force correlation, which vanishes asymptotically, is equal 
to zero. Equation (8.95) shows that the setting of<f^> implies that <f^{t)> = <fo^>, 
this means we thus assume equilibrium conditions. Due to^o ^ «o we specify in 
this way <aQ^> = 2e/[3 rtf]. Correspondingly, the forceXO is characterized by 

{fit)) = 0, (8.96a) 

_M1 /~2\ i,-fl 

{mfin) = ^e -' =^^e '' , (8.96b) 

where the relation <vf > ^ 2e/ 3 is used. In terms of the definition (8.93) oiJ{t) 
and the properties (8.96) we can write the velocity model (8.92) imphed by the 
acceleration model as 

y i (/(0/(^^)) ks) - {v)) ds + fit) . (8.97) 



We see that the force /(O does control both the generation of fluctuations and the 
relaxation. This equation represents a non-Markovian velocity model because the 
future velocity is calculated in terms of the velocity history between zero and t. 

Model Comparison. What is the difference between the Markovian velocity 
model (8.89) and non-Markovian velocity model (8.97) with regard to the process 
statistics provided by these models? The mean velocity is constant in both models. 
Consistent with the consideration of stochastic forces we consider equilibrium 
conditions. For this case, the velocity variance is independent of t, this means we 
have <v^it)>^<v^> = <v^> = 2el3 in both models. Therefore, the difference 
between both velocity models will be given, first of all, by the velocity correlation 
ftinction <vit)viV) >. This correlation function will be calculated and compared 
in the following sections. The difference between the two models also can be seen 
regarding the acceleration correlation. It is interesting that the acceleration corre- 
lation function is determined by the velocity correlation for the equilibrium condi- 
tions considered. To show this we consider 

——{vit)vit + r)) = —{vit)ait + r)) = {ait)ait + r)) + {vit) ^ ^^ 

(8.98) 
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Here, r can be a positive or negative time interval. The left-hand side equals zero 
under equilibrium conditions because all statistics are independent of t (this means 
constant). By writing in the last term the derivative by ? as a derivative by r we 
find the condition 

{a{t)a{t + r)) = -j-^{v(t)v(t + r)). (8.99) 

This finding enables the calculation of the acceleration correlation for any given 
velocity correlation. 



8.5.2 Markovian Velocity Model Analysis 

Velocity Correlation. The statistical properties of Eq. (8.89) can be obtained 
by using the results obtained for the linear stochastic model (8.73). By replacing 
D in Eq. (8.85) by Z) = 4e r/3 we find (r can be positive or negative) 

{v{t)v{t + r)) = ^e^ +yiv,')-^Y ^ . (8.100) 

The last term has to be equal to zero for the equilibrium conditions considered. 
Thus, we set <!;(, > = 2e/3. The use of this setting in Eq. (8.100) then shows that 
<v'{t)>^ <V(,' >, which corresponds to the equilibrium considered. The normal- 
ized correlation function 

CvV^n= ' , , (8.101) 



is then given by the expression 

|r| 

C,it,r) = e'^. (8.102) 

It is interesting that the normalized correlation function of an equilibrium process 
represents a correlation coefficient. For example, for our case we have 

Therefore, there is the requirement that |C„(?, r)| < 1. The normalized correlation 
function (8.102) shows that the condition \CXt, r)] < 1 is satisfied. The function 
C„(?, r) can be used to calculate a time scale r„ that characterizes the correlation 
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time. This integral time scale T^ is defined by the integral 

T^=]cSt,r)dr. (8.104) 



According to Eq. (8. 102), T,. is given by 



r„ = je ^ dr = -T 



/ r 1 \ 
r 



exp 



V 



T. (8.105) 



This relation shows that r determines the correlation time of velocity fluctuations. 
Acceleration Correlation. The acceleration correlation implied by the velocity 
model can be calculated by applying Eq. (8.99). We obtain 

(«(0«(? + r)) = -j^{v{t) v{t + r)) = - Y^e"^ = "1^^ ^- (8-106) 

This consequence of the Markovian velocity model leads to questions, e.g., about 
the negative acceleration variance -2e/ [3 r^] predicted by Eq. (8.106). The result 
(8.106) is implied by Eq. (8.99), which requires the assumption r ^ 0. To see the 
validity of Eq. (8.106) for all values of r we have to calculate the acceleration cor- 
relation directly from the Markovian velocity model (8.89). As shown in exercise 

8.5. 1, this leads to the result 

{d{t)d{t + r)) = -^e^ +^S{r). (8.107) 

it' iT 

Evidence for the need to involve the delta function in addition to the exponential 
function provided by the correlation function (8.106) can be obtained by proving 
thatEq. (8.107) implies 

\{d{t)'5{t + r))dr = Q, (8.108) 



which is a property of any variable (which is here the acceleration) that represents 
the derivative of an equilibrium process (which is here the velocity): see exercise 

8.5.2. This property means that accelerations have a zero integral correlation time. 
The difference between the correct acceleration correlation (8.107) and the incor- 
rect acceleration correlation (8. 106) shows that the use of Eq. (8.99) is problematic 
if the velocity correlation is not a smooth function of r, as given for the model 
(8.102) at r = 0. The acceleration correlation function (8.107) does not lead to a 
finite acceleration variance at r = 0. Hence, a normalized acceleration correlation 
function and a related integral time scale cannot be calculated. 
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8.5.3 Non-Markovian Velocity Model Analysis 

Velocity Correlation. The velocity correlation implied by the non-Markovian 
velocity model is calculated in the last paragraph (called appendix) of this section. 
For the equilibrium considered we obtain according to Eq. (8. 128) the result 



l^{t)v{t + }-))-- 



2e r, e -r^e 



(8.109) 



The parameters r, and r^ are given by the expressions 
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l + Jl-4- 
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These parameters have the properties 
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4r, 



1- 1-4- 



(8.110) 



(8.111) 



which we will use below. The setting r = in Eq. (8.109) reveals <v^'{t)>^2el3. 
Thus, the normalized velocity correlation function (8.101) reads for this case 
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r, e - r, e 



(8.112) 



The integral time scale T„ provides for this velocity correlation function 



T^=\cXt,r)dr = \ 
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(8.113) 



The exponential functions become zero at infinity: Vi and Tj are negative because 
the square root in the definitions (8.110) is smaller than one. The apphcation of 
Eq. (8.1 1 1) implies 7'„ = r, which is the same result as obtained for the Markovian 
velocity model (see Eq. (8.105)). 

Acceleration Correlation. Based on the velocity correlation we can calculate 
the acceleration correlation. In terms of Eq. (8.99) we find 



{a{t) a(t + r)) = -—^ {U(t) Uit + r))-. 



2e d r, e 



-r, e 



2e 



r-,\i-\ I- 

r, e " - r, e 



r, - r. 



3 dr Tj - r2 

_ 2e Tj e - Tj e 
3rr, r, -r. 



(8.114) 
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where Eq. (8. 1 1 1) is used for r, rj. By setting r = we get < a (?) > = 2 e / [3 r Tf]. 
In correspondence to the normahzed velocity correlation function (8.101) we can 
define a normalized acceleration correlation function by the expression 



CM^r)-- 



(a(t)a{t + r)) 



a(t) 



(8.115) 



According to Eq. (8. 1 14) and < a ^(?) > = 2 e/ [3 r r^] we find that 



(8.116) 



The acceleration integral time scale T^ can be calculated in correspondence to the 
calculation of the velocity integral time scale T^ by the relation 



T^^\cXt,r)dr. 



(8.117) 



In terms of Eq. (8.116) and the fact that r\ and r^ are negative we find 



T.=\ 



r, e 



-dr ■■ 



I^(,. 



0, 



(8.118) 



which means that accelerations have a zero integral correlation time. 

Appendix: Velocity Correlation Derivation. Let us show how the velocity 
correlation (8.109) implied by the non-Markovian velocity model can be obtained. 
First, we have to calculate the instantaneous velocity fluctuation that is implied by 
the acceleration model (8.90). Equation (8.90) represents a linear second-order 
differential equation that is driven by the time -depending derivative of the Wiener 
process. By following the solution approach described in Chap. 7 we find 



U{t)- 



1 



4e [ dW 

-— r, r, (r, +r,)\g(t-s) —— (s) ds + d^ g(t) - v^ h(t) 
3 ds 



(8.119) 



The parameters r, and Tj, which are defined by Eqs. (8.110), are the roots of the 
characteristic equation of the linear second-order differential equation. The func- 
tions g{t) and h{t) in Eq. (8.119) are abbreviations that are given by 



jiO- 



h(t): 



r.e' - r, e 



(8.120) 



The initial acceleration and velocity fluctuations a^ and Vq are considered to be 
uncorrelated. The latter assumption is correct for an equilibrium process because 



(flo!^o) = (u(0-^(0 



d(v\t) 
dt 



0. 



(8.121) 
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According to Eq. (8.119), the velocity fluctuation at another time t is given by 
1 



W) 



4e ' dW 

— r, r, (r, + r^)\g{V-s')--{s')ds' + a, g(f) - v, h(r) 
3 ds 



(8.122) 
By multiplying Eqs. (8.119) and (8.122) and taking the ensemble average we find 



{v{t)v(n)- 
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y 'v'2 ('-1 + '-2 ) e + («o' > g (0 g(o + (^o' > ^ (0 KO 



('"2-1) 

where the abbreviation Q is given by 

Q = \\g{t-s)g{t'-s')[--{s)--{s'))ds'ds. 
00 \ds ds' I 

This integral can be evaluated by using the properties ofdWIdt, 

Q = jjg(t - s)g{f-s')S{s'-s)ds'ds =\g{t- s)g{f-s)\ ^\ ' ds'ds 

Q as 

t min(f,f') 

= \g{t- s)g{t'-s)[9{t'-s) - 9{-s)] ds = \g{t- s) git's) ds. 



(8.123) 
(8.124) 



(8.125) 



The calculation of this expression leads to the result 
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\s-\-nm{tX) 



r, r^ g(t - s)gif-s) + h(t - s)h{t'-s) 
(r. -r,)hi\t- 1'\) - r, r, g(t) gjt') - h(t) h{f) 



(8.126) 



where the relations g^(O) = and h(0) = r, - Tj are used. In terms of this expression 
we can write the correlation function (8. 123) as 



{Uit)vin)- 
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(8.127) 



Under the equilibrium conditions considered the last four terms cancel because of 
<Vg > = 2e/3 and <ao^> ^ 2 e / [ 3 r r^ ] = 2 r 1 rj e / 3 . Correspondingly, the velocity 
correlation is found to be given by 



{v(t)v(n)- 



- rjlr-r'l r,\t-t'\ 

le r^e —r^e 



(8.128) 
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Fig. 8.2. The first row shows the normalized velocity correlation function C„ that 
is implied by the non-Markovian {solid line) and Markovian {dashed line) veloc- 
ity model. The velocity time scale t= \, and the ratio if I ris given in the plots. 
The second row shows the derivative of the normalized velocity correlation 
function for the non-Markovian {solid line) and Markovian {dashed line) velocity 
model. 



8.5.4 The Relevance of Memory Effects 



What is the difference between the Markovian velocity model (8.89) and the 
non-Markovian velocity model (8.97) regarding the process statistics provided by 
these models, this means what is the relevance of memory effects? Let us compare 
the acceleration and velocity correlations of both models to address this question. 

Velocity Correlations. The velocity correlation functions (8.102) and (8.112) 
are illustrated in Fig. 8.2. It can be seen that the Markovian and non-Markovian 
models show a similar behavior: there is only a very minor difference between the 
curves for the range r/V r< 0.2 considered (we have the condition r// r< 0.25 due 
to the definitions of ri and r,). The area below the curves is equal to z"= 1 for all 
the cases considered. The effect of the model on the velocity correlation can be 
better seen by looking at the derivative of the normalized velocity correlations. 
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Fig. 8.3. The normahzed acceleration correlation function C„ implied by the non- 
Markovian velocity model is shown in the upper row. The corresponding function 
C^ is shown in the lower row for the non-Markovian velocity model (solid line) 
and the Markovian velocity model {dashed line). The velocity time scale t= \, 
and Tfl ris given in the plots. 



Regarding the velocity correlation implied by the non-Markovian and Markovian 
velocity model, respectively, we find 



dC, 



'■ii'i 



d{r/T) 



■ r, r, T- 
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dirlr) \r\ 



(8.129) 



The corresponding features of the derivatives of normalized velocity correlation 
functions are also shown in Fig. 8.2. The non-Markovian velocity model implies a 
smoothly changing velocity correlation, which has a continuous derivative. This 
behavior is supported by consequences of the Navier-Stokes equations (Sawford 
1991, Pope 1994). In contrast, the Markovian velocity model leads to a velocity 
correlation with a derivative that jumps at r = 0. This unphysical behavior is im- 
plied by the neglect of acceleration correlations (the neglect of a nonzero Tj). 

Acceleration Correlations. The normalized acceleration correlation function 
C„ that is implied by the non-Markovian velocity model is shown in Fig. 8.3 for 
different tj- 1 r, where t= \. Such curves cannot be obtained from the Markovian 
velocity model because the normalized acceleration correlation function cannot be 
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defined for this case: see the discussion of Eq. (8.107). We see that the condition 
|C„(?, r)| < 1 (see the corresponding discussion regarding C„) for the normalized 
correlation function of an equilibrium process is satisfied. Due to the fact that the 
integral acceleration time scale T^ is equal to zero according to Eq. (8.118) we find 
negative values of the normalized correlation function C„. These features can be 
compared with the consequences of the Markovian velocity model by consider-ing 
the following normalized acceleration correlation 

cAur)= ^ \ 1 J' - (8.130) 

2e/(3r ) 

Here, r^ in the definition (8. 115) of C^ is replaced by v. The corresponding curves 
are shown in Fig. 8.3 for both velocity models. For the non-Markovian velocity 
model the C^ curves are similar to the behavior of C„. It may be seen that Cj of 
the non-Markovian velocity model converges to C* of the Markovian velocity 
model in the limit Zf — > co. First of all, the difference between the velocity models 
is given by the acceleration variance <a~(t) >, which is 2 e / [3 r T/] and infinity for 
the non-Markovian and Markovian velocity models, respectively. 

Summary. These observations can be summarized in the following way: The 
neglect of memory effects, which means the use of the Markovian velocity model 
instead of the non-Markovian velocity model, corresponds to the assumption that 
the characteristic correlation time Tf of stochastic forces is negligibly small. This 
approach is equivalent to the consideration of velocities over time steps that are 
large compared to Tf, i.e., the real process is described only asymptotically in this 
case. The latter approach provides velocity correlations that are very close to the 
correlations implied by the non-Markovian velocity model. On the other hand, no 
attempt is made to represent acceleration correlations in a physically correct way: 
these correlations are only represented such that the integral over the acceleration 
correlation function is equal to zero, as required for a variable that represents the 
derivative of an equilibrium process (see the discussion related to Eq. (8.108)). 



8,6 Summary 

The goal of the presentation in this chapter was to develop a methodological 
basis for the modeling of the evolution of stochastic processes. The latter requires 
answers to the questions considered in the introduction, this means: the questions 
about the type of equations for PDFs, their solution, the type of stochastic process 
equations, and the determination of stochastic equations for the modeling of any 
case. Let us summarize the features observed. 
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PDF Evolution Equation. Our starting point was the most general equation for 
the evolution of a PDF given by the Kramers-Moyal equation (8.8), 



df(x,t) _^( 5 I ,„) 



dt „ = 1 1 dx 



D'"\x,t)f{x,t). (8.131) 



The Kramers-Moyal equation represents an identity. We did not use any physical 
principle, we did only assume that the PDFy(x, t) and Kramers-Moyal coefficients 
Z)'"*(x, t) exist. The Kramers-Moyal equation implies Pawula's theorem that shows 
that there are two possibilities: we can either work with an equation that involves 
an infinite number of Kramers-Moyal coefficients Z)*"'(x, i), or we can work with 
the Fokker-Planck equation (8.21), 

dfix,t) _ dD''\x,t)fix,t) ^ d'D''\x,t)f(x,t) ^g^32) 

dt dx dx^ 

which does only involve the first two Kramers-Moyal coefficients. The neglect of 
Z)'"* with « > 3 is justified if the stochastic process considered has a continuous 
sample path, this means if jump processes (i.e., processes involving instantaneous 
unbounded changes) are not considered. We applied the latter assumption in the 
following. We did also assume that the coefficients D''"\x, t) do only depend on x 
and t. This corresponds to the assumption that the stochastic process considered is 
a Markovian process (this means a process for which the present state determines 
the future evolution). 

Solutions of the Fokker-Planck Equation. An important question is how we 
can solve the Fokker-Planck equation (8.132). It was shown that this equation can 
be solved analytically if we consider the specific Fokker-Planck equation (8.34), 

^^-AhO.G(o(x-(x))]/(x,0.^^^^^. (8.133) 

ot ox ox 

The significant difference between the general Fokker-Planck equations (8.132) 
and Eq. (8.133) is that Z)''' is a linear function of x in Eq. (8.133), andZ)*^' is inde- 
pendent of X. It turns out that the solution to Eq. (8.133) is given by a normal PDF 
integrated over the initial condition, 

/(x,0 = j^i=exp|-^^^^|/(x„,?„)A-„. (8.134) 

Here, the model parameter a and /3 are fiinctions of t, and a does also depend on 
Xq. Asymptotically (i.e., for t — >■ oo), a and P relax to the mean <X> and variance 
< X^ > of the process considered. Then, the PDF J{x, t) becomes independent of 
the initial PDF/(xo, 1^): J{x, t) is then given by a normal PDF with mean <X> and 
variance <X~>- 
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Stochastic Process Equations. Instead of asking how the PDF of a stochastic 
process evolves, we may ask how the underlying stochastic process evolves in 
time. In generalization of the stochastic difference equations considered in Chap. 
6, we considered the model (8.55) for the evolution of the stochastic process X(?), 

^(t) = a{X,t)+b{X,t)^{t). (8.135) 

at at 

This approach leads to the question of which PDF transport equation is implied by 
the stochastic model (8.135). To answer this question we calculated the Kramers- 
Moyal coefficients that are implied by Eq. (8.135), which resulted in 

Z)'"(x,0 = fl(x,0, (8.136a) 

D"-\x,t) = -h\x,t). (8.136a) 

Z)"*(x,0 = D^*\x,t) = ■■■Z)'"'(x,0 = 0. (8.136c) 

Hence, the evolution equation for the PDF related to the stochastic model (8.135) 
is a Fokker-Planck equation with coefficients specified through Eqs. (8.136). By 
using the coefficient relations we see that the Fokker-Planck equation (8.133) 
corresponds to the stochastic model 

^(t) = F(t) + G(t){x(t) - {X)]+ ^2m^- (8. 137) 

Hence, a linear stochastic model has a PDF that is a normal PDF integrated over 
the initial condition. The most important advantage of stochastic equations is that 
these equations can be used to represent Fokker-Planck equations that cannot be 
solved analytically. Such PDF evolution equations can be solved by Monte Carlo 
simulation, this means the numerical solution of equivalent stochastic equations 
(see Chap. 6). 

Application to Modeling. How can we determine stochastic process equations 
for the modeling of any case? The stochastic differential equation (8.135) can be 
used for the modeling of any nonlinear processes. On the other hand, Eq. (8.135) 
describes a Markovian stochastic process, and this assumption is often not rigor- 
ously satisfied (most real processes do represent non-Markovian processes). Thus, 
there is the question about the suitability of modeling a non-Markovian process in 
terms of a Markovian stochastic differential equation. To address this question we 
compared in Sect. 8.5 a non-Markovian with a Markovian velocity model: the 
more accurate non-Markovian velocity model (which may be seen to represent the 
reality) was used as a reference model to evaluate the performance of the less 
accurate Markovian velocity model (which represents an approximate model for 
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the real process). It was shown that the Markovian velocity model is not incorrect 
but only less complete than the non-Markovian model. The latter model describes 
processes that take place over the time scale z} (over which accelerations change) 
and over the time scale r (over which velocities change). On the other hand, the 
Markovian model does only describe processes that take place over r. The per- 
formance of the Markovian model is acceptable regarding the processes that are 
described: this model provides velocity correlations that are very close to the cor- 
relations implied by the non-Markovian velocity model. It is often only possible to 
model a part of all the processes observed in reality (there are often processes that 
take place over a variety of time scales, which vary over orders of magnitude). 
The application of a Markovian model that provides an accurate description for a 
certain part of these processes and neglects other (smaller-scale) processes does 
often turn out to be the most convenient choice. 



8.7 Exercises 



8.2.1 Show the consistency of the Fokker-Planck equation (8.21) by integrating 
this equation over the sample space from negative to positive infinity. 

8.2.2 Consider the Fokker-Planck equation (8.21). 

a) Calculate the asymptotic solution to this equation (this solution has the 
property dfl dt = 0). 

b) Provide an example for Z)*'' and Z)*^' that leads to a PDF that approaches 
zero for |x| — >■ co. 

c) Provide an example for Z)*'' and Z)'^' that leads to a PDF that diverges for 
positive or negative x with |x| — >■ co. 

8.2.3 Consider the Fokker-Planck equation (8.21). Specify this equation (deter- 
mine the coefficients of the Fokker-Planck equation) so that 



yjlTia [ 2(7 

represents an asymptotic solution. The normal PDF parameters jU and crare 
considered to be constant. Hint: you may assume that Z)'"' is constant. 

8.2.4 Consider Eq. (8.25) for the mean <X>, which is implied by the Fokker- 
Planck equation (8.21). Try to solve this equation for the case that Z)'''(x, t) 
= -ax^, where a is any constant. 
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8.2.5 Consider Eq. (8.32) for the variance <X^ >, which is a consequence of the 
Fokker-Planck equation (8.21). 

a) Solve this equation for the case that Z)'''(x, t) ^ -ax and D'^^ is constant. 
Here, a is any constant. 

b) Find the asymptotic variance according to this equation. 

c) Explain the relevance of this result regarding the determination of model 
parameters. 

8.2.6 Consider the Fokker-Planck equation (8.21). 

a) Follow the approach used in Sect. 8.2.3 to find the evolution equation 
for the central moment of third order < X^ >■ 

b) Solve this equation for the case that Z)''*(x, t) ^ -ax and Z)*^' is constant. 
Here, a is any constant. 

8.3.1 Consider the Fokker-Planck equation (8.21). 

a) Follow the explanations in Sect. 8.3.1 to show that the Fokker-Planck 
equation (8.21) applies to the two-point PDF J[x, t; x', t') = <S[x-X{t)] 
S[x'-Xif)]>, 

df{x,t;x\t') ^ dD''\x\t')f{x,t;x\t') d'D^'\x\nf{x,t;^\n _^ 
df 8x' dx'^ 

b) Apply the definitionXx, t; x\ f) = <S[x-X{t)] S[x'-X(f)]> to show that 
the correlation function <X(t)X{t') > is defined by 

X{t)X{t')^ = ll{x - {X)){x'-{X'))fix,t;x',t')dxdx'. 

c) Use the definition of < X{t)X{t + r) >, where r is any non-negative time, 
and the Fokker-Planck equation for the two-point PDF/(x, t; x\ t + r) to 
show that <X(?)X(? + r) > satisfies the equation 



dlX{t)X{t + r) ^ 

-\ '- = {X{t) Z)'" {X{t + r),t + r)^ 

8.3.2 Consider Eq. (8.40) for the conditional PDF j{x, t \ Xq, ?o)- Calculate the 
asymptotic solution to this equation (which has the property dfl 5? = 0). 
Assume for simplicity that G and D are constant and F=Q. 

8.3.3 Consider the model (8.42) for the conditional PDF/(x, t \ Xq, t^. Show that 
the consistency with the initial condition (8.41) requires that a and /?have 
the initial values a{t^ = Xq and P{t^ = 0. 

8.3.4 Consider Eqs. (8.48) for a and /?, 

^ = G{t){a - {X))+ Fit), ^ = 2Git)P + 2D(t). 
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a) Show that the following expressions are the solutions of these equations 
and satisfy the initial values «(?o) " ^o and y6(?o) = 0. 



a = Xg+'\ expJ ' '\G{r) dr i {G{s)[xg - {X)] + F(s)) ds. 



IS + If, 

}( 

J3 = 2\ expi 2 7G(r) driois) ds. 

'0 I '0 J 

b) Specify the solutions for the case that G and D are independent of t and 

8.3.5 Consider the Fokker-Planck equation (8.34) for the case that Z) = 0. 

a) Find the function /5(?) for this case. 

b) Find the solution^^, t) to the Fokker-Planck equation for this case. 

c) Specify the solution y(x, t) obtained in b) for the case that the initial PDF 
is given by/(xo, Iq) ^ S(xq- «o), where «o is a given nonrandom value. 

d) Interpret the result obtained in c). 

8.3.6 We consider an instantaneous emission of a substance from a point source, 
this means the emission of a mass M at time zero at a fixed position H. The 
substance diffuses along the y direction. The mean substance concentration 
is given by C(y, t) = MJ(y, t). Y{eve,J(y, t) refers to the PDF for finding a 
parcel at time ? at a position y (see Sect. 6.3.3). The concentration C is 
described by the diffusion equation (D is a constant diffusion coefficient) 

dC{y,t) _ ^ d'c{y,t) 

dt dy^ 

a) Specify the initial concentration Ciy^, 0) for this case. 

b) Calculate the solution to the diffusion equation based on the solution of 
the Fokker-Planck equation (8.34). 

8.4.1 Consider the correlation function (8.87) and variance (8.88). 

a) Show that Eq. (8.87) agrees with the consequence (8.33) of the general 
Fokker-Planck equation. 

b) Show that Eq. (8.88) agrees with the consequence (8.32) of the general 
Fokker-Planck equation. 

8.4.2 Consider the case that you are interested to use the linear stochastic model 
(8.73) for the modeling of a case considered. 

a) How is it possible to determine the parameters D and r of the model 
(8.73) in terms of measured statistics? 

b) How is it possible to provide evidence for the suitability of modeling a 
certain case in terms of a linear stochastic model? 



8.7 Exercises 333 



8.4.3 The mixing of species in water or air can be described by the model 

Here, ^ refers to the instantaneous mass fraction, which is bounded by zero 
and one according to its definition, i.e., < ^ < 1. Here, <(zS> is the mean 
value of ^, r is a characteristic mixing time scale, c is a parameter, t is time 
and dWI dt refers to the derivative of a Wiener process. 
For simplicity, we assume that r, c, and <^ are constants. 

a) Use the stochastic mixing model to derive the equation for the variance 
<(j)^>. Solve this equation. 

b) Use the stochastic mixing model to derive the corresponding equation 
for the standardized species mass fraction 0= {(f)- <({>>) l<^^ >"^. 

c) Use the equation for to discuss the consequences of applying a zero 
model parameter c. Relate this discussion to the solution of the variance 
equation. 

d) Use the stochastic mixing model to discuss the disadvantage of using a 
nonzero c. Hint: consider the property < (zi < 1 of (zi. 

8.4.4 Continue with exercise 8.4.3. The PDF /(^, t), which is related to the sto- 
chastic model considered, is given by 

f{e,t)^\f{e,t\ff,f)f{e\f)de\ 

Here,X6', ?| e\ t) and/(6'', f) refer to the conditional PDF and initial PDF, 
respectively. 

a) Provide the evolution equation and initial condition for the conditional 

vmAO,t\e',t'). 

b) Solve this PDF evolution equation. Provide all the model parameters of 
J{9,t\9\ f) as explicit functions of time. 

c) Calculate/6i, t) for the case ihaifiO', f) ^ S{0' - <<p>). 

d) Describe qualitatively the evolution o{J[6, t) obtained in this way. 

8.4.5 Consider the stochastic population model discussed in Sect. 6.5, 

dP , „/ dW 

— = P[\- P)\ u + a 

dt ^ \ dt 

a) Use Eqs. (8.25) and (8.32) to obtain the equations for <P> and <P^ >. 

b) Show that the discrete Eqs. (6.94) and (6.96) derived in Chap. 6 imply 
for A? — > the same equations for the mean and variance of P. 

c) Present the corresponding equation for the PDFy(p, i). 

d) Explain why the equation obtained in c) cannot be solved analytically. 
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8.5.1 Consider the Markovian velocity model (8.89). 

a) Use the velocity model to show that the acceleration correlation function 
is given for all values of r by the expression 

{a{t)a{t + r)) = -^e ' +i^5{r). 

b) Show that the integral of this correlation over < r < oo is equal to zero. 

8.5.2 Show the validity of 

\(^{t)a{t + r))dr = Q 



for any variable a that represents the derivative of an equilibrium process. 
Hint: perform the integration over a{t + r) = dv{t + r)l dr directly. 

8.5.3 Consider the velocity correlation function (8.109), which is implied by the 
non-Markovian velocity model. Calculate the limit r^ — > of this velocity 
correlation function to recover the normalized velocity correlation function 
(8.102) of the Markovian velocity model. 

8.5.4 Consider the acceleration correlation function (8.114) implied by the non- 
Markovian velocity model. Calculate the limit if — > of this acceleration 
correlation function to recover the acceleration correlation function (8.107) 
of the Markovian velocity model. 

8.5.5 The velocity model (8.89) implies the following model for the position x 
defined by dxl dt = v. 

Here, <u> is constant, and the stochastic force F(t) is defined by 

i7/^\ ~/-n\ -"r , 1^^ r -{i-s)ir dry , , J 
F(t)=v{u)e +J — \e (s) ds. 

V 3z" ds 

a) Show that this position model agrees with the velocity model (8.89). 

b) Explain why the position model represents a non-Markovian model. 

c) Calculate under equilibrium conditions the mean and correlation of the 
stochastic force 7^(0. You may follow the explanations in Sect. 8.5.1. 

d) Which condition does this non-Markovian position model reduce to a 
Markovian position model? To provide the answer to this question you 
have to specify the force F{t). Use the relation e = 3 v/ r between e and 
the kinematic viscosity v (see Sect. 10.5). Neglect the first term in the 
F{t) expression, which is justified under equilibrium conditions. 
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The discussions of deterministic evolution in Chap. 7 were focused on the 
modehng of the evolution of one variable (as heat, mass, the position of any body, 
or a population density). The consideration of such relatively simple problems is 
helpful for a basic understanding of the structure and the range of applicability of 
equations for typical problems. However, only a narrow range of problems can be 
described in this way: the analysis of most real problems requires the considera- 
tion of the multivariate evolution of several variables. The latter is required, for 
example, regarding the interaction of biological species and motions of bodies or 
fluids in three-dimensional space. To deal with such cases we extend here the 
concepts used for the modeling of mechanical and population ecology processes in 
Chap. 7 to the modeling of the joint evolution of several variables. We will 
continue with the consideration of global properties that change in time but not in 
space, i.e., partial differential equations that describe the evolution of processes in 
space will be not considered. The mathematics of models for the evolution of such 
processes can be formulated in terms of linear and nonlinear systems of coupled 
ordinary differential equations. 

Section 9. 1 explains the motivation for developing mathematical models for the 
multivariate evolution of processes. Section 9.2 prepares the discussions in the 
following sections by the explanation of techniques for the solution and analysis 
of coupled systems of ordinary differential equations. Sections 9.3 and 9.4 extend 
the discussion in Chap. 7. Section 9.3 describes the modeling of basic population 
ecology processes (the competition for food and predator-prey interactions). The 
modeling of mechanical motions will be considered in Sect. 9.4, where the pendu- 
lum equation used in Chap. 3 will be solved. Section 9.5 illustrates the problem of 
dealing with the fluid dynamics equations derived in Chap. 10 by considering a 
simple model for atmospheric motions. Section 9.6 summarizes the basic features 
of the modeling approaches presented in this chapter. 
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r r 

10 15 20 



Fig. 9.1. A solution >>!(?) of the Lorenz equations 
(9. 1) combined with the model parameter R = 28. 
The thick line and the thin line present solutions 
for the initial values (>'io,>'20; J30) " (5, 5, 5) and 
Oio, J20, J30) = (5.01, 5, 5), respectively. 



9,1 Motivation 



Weather Forecasting. Weather forecasting is crucially relevant. Weather warn- 
ings are used to protect life and property. Temperature and precipitation forecasts 
are highly relevant to agriculture. Regarding everyday life, weather forecasts are 
used to find out what to wear on a particular day. Weather forecasting has to be 
performed of the basis of numerical solutions of complicated equations (systems 
of nonlinear coupled partial differential equations) that involve several variables 
(as the three velocity components in space and temperature). To see basic features 
of such systems of partial differential equations (like the weather predictability), it 
is helpful to consider highly simplified approximations to these equation systems 
- as given by the Lorenz (1963) model. The latter model is given by the equations 



(9.1a) 
(9.1b) 
(9.1c) 



Here, jv, measures the strength and direction of atmospheric circulation, andj2 and 
J2 measure the horizontal and vertical temperature variation, respectively. The 
variable that essentially controls the dynamics of this equation system is R, which 
is proportional to the vertical temperature difference. Two solutions to these equa- 
tions, which differ by a minor difference of the initial value for j,, are shown in 
Fig. 9.1 (details about the Lorenz equations (9.1) and their numerical solutions can 
be found in Sect. 9.5). This figure illustrates that solutions of the Lorenz equa- 
tions reveal a complicated behavior. Also, small variations of initial conditions 
may result in completely different solutions - which indicates that long-range 
weather forecasting may be impossible. 



dt 


= 10(^2 -Ji), 


dy, 
dt 


= y,iR-yi)-y2 


dy, 
dt 


8 

= yiy2--yy 
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Questions Considered. The analysis of solutions of the Lorenz equations (9.1) 
leads to questions like: 

• How can we determine different (chaotic and nonchaotic) solution regimes? 

• How can we analyze the influence of variations of initial conditions? 

• How can we characterize the asymptotic behavior of solutions? 

The Lorenz equations do only represent one example for many problems that have 
to be described by (linear and nonlinear) systems of coupled equations for several 
variables. Thus, from a more general point of view there are questions like: 

• How can we formulate laws for the multivariate evolution of several variables? 

• Do all multivariate evolution equations have (convergent numerical) solutions? 

• How can we analytically study multivariate evolution equations? 
The latter and other questions will be addressed in this chapter. 



9,2 Systems of First-Order Differential Equations 

Techniques for the solution of linear systems and the analysis of basic features 
of solutions to nonlinear systems of first-order ordinary differential equations will 
be described in this section. This discussion will provide an appropriate basis for 
the developments to be performed in the following sections of this chapter. 



9.2.1 Linear Systems of First-Order Differential Equations 

Equations Considered. The analysis of linear systems of ordinary differential 
equations is helpful because of two reasons: many problems (like the linearized 
pendulum equation: see Sect. 9.4.2) can be solved by linear equations, and linear 
equation systems can be used to analyze the solution features on nonlinear equa- 
tion systems (like the Lotka-Volterra equations and Lorenz equations: see Sects. 
9.3.3 and 9.5.2, respectively). Thus, let us consider the following linear equation 
system, 

^ = fliiJi+«i2J2. (9-2a) 

at 

— ^ = 021 Ji + 02272 ■ (9-2b) 

at 

Here, a^, fli2, fl2b and 022 are constants. The solution of the equation system (9.2) 
requires initial values ji(0) = jio and ^2(0) = J20; where jio and J20 are considered to 
be given parameters. 
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Relation to Second-Order Equations. A good way to find the solutions yi{t) 
and ^2(0 to the equation system (9.2) is to exploit the relationship between this 
equation system and second-order differential equations. This relationship can be 
derived in the following way, 



d^y. dy, dy^ dy. . , 

dt' dt dt dt 



dyi ( dy^ ] j \dy^ 

= «ii -^ + «i2«2iJi + «22| "^ " "ii^'i J = Ki + "22^"^ " ("""22 - ai2«2i) Ji- 

(9.3) 
In the first line, Eq. (9.2b) was applied to replace dyj I dt. In the second line, we 
used Eq. (9.2a) to replace Oij j2- In the same way we find for d^y2l di^ 

d^y^ dy. dy2 , . dy^ 

—TT- =0^^—^+ 022 —r = «21 (fluJ'l + «12>'2) + «22 "J^ 

dt" dt dt dt 

c/v, f dy^ \ / \dy2 , N 

= ^22— + ai2a2iy2 + "ill "^ " "22 J2 J = l«ll + "22 j-^ - («11«22 - «12«2l) J2- 



(9.4) 



Equations (9.3) and (9.4) can also be written 



^.b^.cy,=0, (9.5a) 

dt dt 

^ + h^ + cy2=0, (9.5b) 

dt dt 

where the following abbreviations are applied, 

Z? = -(«[,+ 022)' C = 011022 -fli2 '221- (9.6) 

Equations (9.5a) and (9.5b) represent the same equation. Different solutions yt{t) 
and 3^2(0 of these equations are obtained by applying the initial values j^O) = Jk, 
and 3^2(0) ^ J20' ^nd the initial derivatives dyi I dt{0) = j^'^ and dy2 1 dt(0) = j'20 that 
are provided through Eq. (9.2). 

yio=«iiJ'io+«i2>'2o> (9-7a) 

y20=«2lJlO+«22J20- (9-7b) 

Equations (9.5) correspond to the second-order equation (7.45). Accordingly, the 
solution of Eq. (7.45) derived in Chap. 7 can be used for the solution of the equa- 
tion system (9.5), as will be shown in the next paragraph. Before doing this we 
will show that the relationship between a linear system of first-order equations and 
a linear second-order differential equation also can be used to write a linear 
second-order equation in terms of a system of first-order differential equations. 
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The latter can be seen by considering Eq. (7.45), 

a^^b^ + cy = Q. (9.8) 

dt^ dt 

We set ji =3; and j2 = dyldt. Differentiation of ji and jj then provides 

^-'^-y., (9.9a) 

dt dt 



dy^ d~y b dy c c h 



dt dt a dt 



y = —y, — y„ (9.9b) 



a a 



where Eq. (9.8) was applied. Hence, the second-order differential equation (9.8) 
can be represented as the system (9.9) of first-order equations. The initial values 
required for the solution of Eq. (9.9) are given by the initial values j,(0) ^^(O) and 
3^2(0) ^ dy/dt(0) that complete the second-order equation (9.8). 

Solution of the Equation System. I. The solutions to Eqs. (9.2) can be derived 
via the solution (7.57) of the corresponding second-order equations (9.5). For 
a = 1 we obtain according to Eq. (7.57) 

y^^AoZllIiiL/^' .yjoZlLliL/^', (9.10a) 



y 10 hy\o ^m 


>'lO~''l J'lO ^'-2' 


ri-h 


ri-rj 


y'lo-hyio ^'-^^ 


' y'lo-fxyiD ^•■2' 



1 - ''2 '1 - '"2 

The initial derivatives/ 10 and/jo ^re given through Eqs. (9.7). The eigenvalues /'i 
and Tj are determined by the characteristic equation 

Q-r -{a^^+a2^)r + a^^a^^-a^^a2^-{a^■^-r){a22-r)-a^2a2^, (9.11) 

which follows from the use of Z? = -(«!, + 022) ^^d c = Au ^22 ~ «i2 <^2\ in the 
characteristic equation r^ + br + c = Q. Hence, the eigenvalues i\ and r^ are 

Ti^r^+r^, r^^r^-Yj,, (9.12) 

where r^ and r^ are given by 

,_fll±^^ (9.13a) 

'■fl =^v(«ii +«22)^-4(aiifl22-«i2«2i) =-V("ii -«22)^ + Aa^j^ci^^. (9.13b) 

Solution of the Equation System. II. Let us directly solve the equation system 
(9.2) to check the validity of the solutions (9.10). Such a solution can be found 
efficiently by making use of vector and matrix notation. We write Eq. (9.2) as 

^ = Ay. (9.14) 

at 
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Here, the vector j and the matrix yi are given by 



A = 



a, I 



(9.15) 



•'22 7 



To solve Eq. (9. 14) we assume according to Eq. (9. 10) an exponential solution, 



where the constant vector c is given by 



The use of Eq. (9.16) in Eq. (9.14) then results in 

rt J rt 

rce = Ace . 
Upon cancelling the nonzero exponential function we obtain 

{A-rl)c = 0. 
Here, I is the 2x2 identity matrix. 



I 



1 
1 



(9.16) 

(9.17) 

(9.18) 
(9.19) 

(9.20) 



The identity matrix I has the property \c = c. The inverse matrix of ^ - r I will 
exist if the determinant dsi{A - r I) is nonzero. In this case, we do only obtain 
trivial solutions c = according to Eq. (9.19). Thus, the condition to obtain non- 
trivial solutions c is given by det(.4 - r I) = 0, i.e.. 



det(^-rl)^ 



a, 



- r 



a, 



fl. 



0. 



21 "22 

The latter constraint can be also written 



= (o,[ -r){a22 -r) - a^ flji = '' ~('^ii + ai-^)^ ^ '^u'^21 ~'^n'^2i 



(9.21) 



(9.22) 



The solution of this quadratic equation for r reveals that the two eigenvalues I'l 
and Tj obtained in this way agree with the eigenvalues r, and r, given by Eq. 
(9.12). To specify the solution (9.16) we have to use the eigenvalues r, and rj in 
Eq. (9.19) to determine the corresponding eigenvectors c''' and c'^'. This constraint 
provides the equations 



Uf, 



^C,^<" 



ly V 2 



•-I 



(9.23a) 
(9.23b) 
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The conditions for the first eigenvector c ' can be written 
= («ii -''i)Ci*"+a,2C2"', 
O^fljjC, +(a22-r,)c2 . 



Equation (9.24a) can be used to express C2' in terms of c/ ', 



c. 



(1) 






(9.24a) 
(9.24b) 

(9.25) 



The use of this expression in Eq. (9.24b) provides a relation for c/'', 

1 



0: 



-(a,i-ri)(a22-r,) 



c/" =[(«ii-l)(fl22-'i)-«i2«2i]ci*"- (9.26) 



The eigenvector r, solves the characteristic Eq (9.22). Thus, the bracket term in 
the latter relation is equal to zero, which means that there is no constraint on c/''. 
Hence, we may assume that c/" = Cj, where Cj is an open parameter. The eigen- 
vector c'" can be written then c'" = Cj «i, where 



1 



(9.27) 



*12 J 



The corresponding condition for the second eigenvectors c'^' can be derived in the 
same way. According to Eq. (9.23b) we have 



= («[, -r2)C[ + fl,2C2 " , 

= a,,C[ + (022 - ''2 ) '^2 ■ 



(9.28a) 
(9.28b) 



Using the relation for c,'"' implied by Eq. (9.28a) in the second relation leads to 



0: 



- + a. 



a, , - A 



C2*^' =[(«ll-'"2)(«22 -'2)-«12«2l] ^2 



(2) 



(9.29) 



This relation does not imply a condition on C2**' because the eigenvalue r, satisfies 
the characteristic equation (9.22). By setting Cj*^' = C2, where C2 is an open param- 
eter, we find by means of Eq. (9.28a) 



.(2) 



«n -r. 



(9.30) 



Hence, c ' can be written c'' = C2 Mj, where 



(9.31) 
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By accounting for the two possible exponential solutions we can write the solution 
of the equation system (9.14) as 



J = Ci H, e +c^u^e- , 
which means according to Eqs. (9.27) and (9.31) that 



V 


1 ^ 


e'> 




f 


«12 J 


V 



1 



p'. 



(9.32) 



(9.33) 



This solution has to satisfy the initial conditions, which implies two equations for 

Cy and c-i. 



Jio = c. 



au-r-, 



yi. 



(9.34) 



The use of the second relation for c^ in the first condition, and the use of the first 
relation for c^ in the second relation imphes 



J^lO = Ci 



J'20 



flp 






-J20, (9.35a) 



3^20 



a, , - r, 



>'io+- 






"11 '2 
Correspondingly, Cj and Cj are given by 

^ _ Jlo(«ll -^2) + «12>'20 _ y 10 -^2 3^10 
c, - - 

h - '2 '1 - ''2 

C2 = — 7 — ^[«i2 J20 + («ii - '•i);'io] = 



f \ 

y au-''i 

V ^11 ~ ^2 y 



Op 



-J,o- (9.35b) 



«ii-^2 yiQ-'\yu 

«12 1 - ''2 



(9.36a) 
(9.36b) 



where the definitions (9.7) of initial derivatives are used as abbreviations. With 
these expressions we can write the solution of Eq. (9.14) 



yA y'lo-fiyu 



f 1 ^ 

a,, -r, 



yi ''i-'2 



,''i' y \o 1 Jio 



f 1 ^ 

a, I -r. 



(9.37) 



Consistency of Solutions. This solution provides ji as given by Eq. (9.10a). To 
show that J2 given by Eq. (9.37) agrees with Eq. (9.10b) we do the following: The 
initial value J20 and initial derivative j'20 implied by Eq. (9.37) are given by 



J20 



P-Q 



_ r,P-r,Q 

y 20 

r, -r. 



(9.38) 
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Table 9.1 Cases considered for the illustration of solutions of the linear equation system (9.42). 

^11 ^12 ^21 ''l ^1 

Case 1: Real unequal eigenvalues of the same sign -1 0.5 0.5 -0.5 -1.5 

Case 2: Real unequal eigenvalues of opposite sign -12 2 1-3 

Case 3: Real equal eigenvalues -1 -1 -1 

Case 4: Complex eigenvalues -1 1 -1 -1+i -1-i 

Case 5: Pure imaginary eigenvalues 1 -1 i -i 



Here, we used the abbreviations 

^ = -^^^^^(y,o-'2j,oX e=-^^^^^(yio-'-,j,o)- (9.39) 

The use of P = {i\ - r^yjo + Q and (? = -P - (''i - '"2)>'2o according to Eq. (9.39) 
enables us to write 

r, - Tj r, - r. 

The latter two relations between Q and P with j'20 can be used to obtain 

y2o-'-ij2o = e = -^^^^^^(yio-'-iJio), (9.41a) 

an 
y2o-^2 J20 = ^ = -^^^^^(j'io-'-2 J.o)- (9.41b) 

The combination of these relations with Eq. (9.37) recovers the solution (9.10b). 



9.2.2 Features of Solutions of Linear First-Order Systems 

Example. Let us illustrate some characteristic features of solutions of the linear 
equation system (9.2). For simplicity we assume that a^ = fl22) 

^ = aii>', +ai2>'2, (9.42a) 

at 

^ = a2i>'i+«ii>'2. (9.42b) 

at 

According to Eq. (9. 12), the eigenvalues r, and 1-2 are then given by the relations 



r, = a, 1 + 7«i2«2i' '2 = «i 1 ~ V^iAi'. (9.43) 
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Fig. 9.2. The temporal evolution ofj, and>'2 in the 
yyVi phase plane according to the linear equation 
system (9.42); (a), (b), (c), (d), and (e) show the 
evolution of j/O and jjCO for the five cases given 
in Table 9.1, respectively. Several initial values of 
yy and J2 are considered. 



Five sets of model parameters On, a^, and Oji are specified in Table 9.1. The cases 
considered correspond to five characteristic types of the eigenvalues }\ and r,. The 
solutions of the equation system (9.42) were obtained numerically. The evolution 
ofj'i and>'2 in time is shown in Fig. 9.2 in the>'i-j2 phase plane for several initial 
values jio and Jjq. Such curves can be seen as the trajectory of a particle moving 
with a velocity dyldt = Ay. 

Analytical Solutions. The analytical solutions for the five cases considered are 
given by Eq. (9.32). To prepare the discussion of specific cases we will calculate 
the solution for the first three cases. Neither case 4 nor 5 will be involved here: 
these cases require rewritings of the solution (9.32) to have real-valued solutions 
(see Chap. 7). To cover the first three cases we set fli2 = Oji = £■ Here, ^ represents 
a positive parameter. For the cases 1, 2, and 3 we have the values s = (0.5, 2, 0), 
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respectively. According to Eq. (9.43), the eigenvalues are then given by 



r, =-! + £, 



-l-£. 



Equations (9.27) and (9.31) imply for the eigenvectors 



1 ^ ^ 
a, , - r, 



*ii 



a 



12 J 



1 ^ 
-£ 



V £ J 



1 



£ 

£ 

1 



According to Eq. (9.36), the coefficients Ci and Cj are given by 

^ _ QiiJio + any 20 ' ^i Jio _ -J^io + g J^2o + (1 + s)ym _ J20 + Jio 
1 — — — 

r, -Fr. 2£ 2 



(9.44) 



(9.45) 



(9.46a) 



«ii-'2 «ii>'io + «i2>'2o-'"i>'io _ £ -yio + syio + i^-s)yio _ J20-J10 



2£ 



Thus, the solution (9.32) reads 

'■■' '-ii _ J20 +710'^ 



g-(i-.V ^ J20 JlO |g 



-(!+£)( 



In terms of ji andy2, this solution can be written 

_ 3^20 + JlO „-(!-£)? >'20 ~ ^^lO „-(l+J)' 
•^1 ~ 1 1 ' 



_ J2O + JlO „-(l-£)f , J2O JlO „-(l+£)f 

'2 2 



(9.46b) 
(9.47) 

(9.48a) 
(9.48b) 



To see the relation between j'j on j,, which determines the trajectory in the Ji-j'2 
phase plane, we consider the sum and the difference of these two expressions. 



yi + y^ = (j2o + >'io)e 



-(l-c)t 



yi-yi = (yio-yio)e 



-a+c)! 



(9.49) 



Case 1. The first case considers two real unequal eigenvalues of the same sign. 
For this case (£•=0.5), the solution (9.47) reads 



-0.it , -1.51 



-0.it ( , -t\ 

■■ e yc^ Kj + C2U2e j. 



(9.50) 



Examples for the evolution of j, and jj in time are shown in Fig. 9.2a for different 
initial values. All trajectories are attracted by the equilibrium solution (0, 0). For 
two real unequal eigenvalues that are positive one finds the opposite feature that 
all trajectories increase their distance to (0, 0). The trajectories are aligned with 
the eigenvectors. There is, however, a difference in the behavior of trajectories. 
The term c «2 e ' in Eq. (9.50) is small compared to c Ui for sufficiently large t. 
Thus, the trajectories tend toward h, before they reach the equilibrium point (0, 0). 
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The trajectories are characterized by Eq. (9.49), where £•=0.5. For ? — > oo we find 
jVj = iji, which agrees with the eigenvectors (9.45). 

Case 2. The second case considers two real unequal eigenvalues of opposite 
sign. The solution (9.47) reads for this case 

y = c^ H, e' H-Cjaje"" =e'(c, u^ h-Cj Hj^ "')■ (9-51) 

Examples for the evolution of >>, and y^ are shown in Fig. 9.2b for different initial 
values. The behavior of trajectories can be explained by considering the solution 
(9.51). As given for the first case, trajectories tend toward Hi because the Hj term 
in Eq. (9.51) becomes negligible compared to the Hj term for large t. The differ- 
ence to the first case is given by the fact that the solution tends (due to the positive 
eigenvalue) to infinity after reaching «,. The relation between j, and j2 is deter- 
mined by Eq. (9.49), where s = 2. For ? — > oo this relation provides _)'2 ^ ±yi in 
agreement with the eigenvectors (9.45). 

Case 3. The third case considers two real equal eigenvalues. The solution for 
this case can be found by considering the limit £— > of Eq. (9.47), 



;'20+Jl0|l|_^>'20-Jl0 



3^10 , „-, ^g 32) 

3^20 ; 



Examples for the evolution of ji and jj ii time are shown in Fig. 9.2c. Expression 
(9.52) explains the difference to the cases 1 and 2: trajectories do not tend toward 
Ml because the bracket term is independent of time. The path of trajectories can be 
derived from Eq. (9.52), 

J2 = — Ji- (9.53) 

Hence, every trajectory lies on a straight line through the origin. 

Case 4. The fourth case considers two complex eigenvalues. The eigenvectors 
are also complex. Figure 9. 2d shows that all trajectories tend toward the equilib- 
rium point (0, 0). The simplest way to see in which way the equilibrium is estab- 
lished is to analyze the consequences of the equation system (9.42) directly, 

^ = -Ji+J2, (9.54a) 

at 

^ = -y,-y2- (9.54b) 

at 

We multiply Eq. (9.54a) by 2 j, and Eq. (9.54b) by 2^2, and we take the sum of 
both equations, 

^{y'+y') = -2{y,'+y,'). (9.55) 

at 

This relation was obtained by making use of the identity dy^ I dt ^2yi dyj dt, and 
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a corresponding relation for j'j- The solution of this equation is given by 

y'+y2=iy^o'+y2o')e-"■ (9.56) 

Therefore, the trajectories represent circles with a radius that decreases in time, 
this means the trajectories are spirals. 

Case 5. The fifth case considers pure imaginary eigenvalues. Figure 9.2e shows 
that the trajectories are given by circles in this case. Evidence for these trajectories 
can be obtained by the consideration of the equation system (9.42) for this case, 

^ = y2, (9.57a) 

at 

^ = -y,. (9.57b) 

at 

As for the fourth case, we multiply Eq. (9.57a) by 2yi, Eq. (9.57b) by 2^2, and we 
take the sum of both equations, 

4(J,'+J2') = 0. (9.58) 

at 

Hence, the trajectories are indeed circles, 

yx^ + yi^ ^ yw^ + yio^ ■ (9-59) 

The critical point (0, 0) is called a center. 

Summary. The features of solutions of systems of first-order linear differential 
equations can be summarized in the following way (Boyce and DiPrima 2009). 
There are three possibilities for the evolution of trajectories: 

a) Trajectories approach the equilibrium point as ? — > oo. This behavior is seen if 
the eigenvalues are real and negative or complex with real negative part. Such 
a system is called asymptotically stable. 

b) Trajectories remain bounded but they do not approach the origin. This behavior 
appears if the eigenvalues are pure imaginary. Such a system is called stable. 

c) Trajectories become unbounded as ? — > oo. Such a behavior is seen if at least 
one eigenvalue is positive or if the eigenvalues have a positive real part. Such a 
system is called unstable. 



9.2.3 Analysis of Nonlinear Equation Systems 

Nonlinear Equation System. The analysis of nonlinear equation systems is 
more difficult than the analysis of linear systems because nonlinear systems can 
hardly be solved analytically. To illustrate the way of analyzing the behavior of 
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nonlinear systems, let us consider an equation system that will be used also for the 
discussion ofpopulation ecology processes, see Sect. 9.3, 



dt 

dy2 
dt 



= Jilai+^iJi+CiJj), 
= y^ia^ + b2y2 + C2yi). 



(9.60a) 



(9.60b) 



Here, Oi, b^, Ci and Oj, b2, C2 are any positive or negative constants. One approach 
to analyze the nonlinear equation system (9.60) will be discussed in the following: 
The idea is to determine the equilibrium points and to analyze the solution suffi- 
ciently close to the equilibrium points such that the nonlinear equation system can 
be approximated by a linear equation system that can be solved. 

Equilibrium Points. An equilibrium solution (7,, Fj) of Eqs. (9.60) is defined 
by Fi and Y2 values so that dyi I dt = dy2 1 dt = Q. Therefore, equilibrium solutions 
are defined by the conditions 

Q = y^{a^+b^y^+c^y2), (9.61a) 

= j2(«2+^2>'2+C2j'l)- (9.61b) 

The equation system (9.60) provides four equilibrium points, which are given by 



(5^,y,) = (o,o), (y„y,)= 



{YuY2) = 



iJiJi)- 



■^2 J 



f 


«1 





V 


h 


J 



fljCj - a^b2 OjCj - 02^1 

y ^1^2 - Cf2 ' ^1^2 ~ C1C2 J 



(9.62) 



The validity of the first three equilibrium points can be easily seen. The last equi- 
librium point ensures that both parenthesis terms are equal to zero. The validity of 
this claim may be proven by using this point in the parenthesis terms of Eqs. 
(9.61a) and (9.61b), respectively. 



«i + hyi + C1J2 =a^+bi 



fljCi - fli^2 



+ c, 



fljCj - a2^i 



1 1 

bib2 -C1C2 btb^ -c^c, 



[1^2 •'i'-2 "\"2 ~ '-i'-2 (9.63a) 

= flj {bfi2 - CjC2 ) + ^1 (02^1 - 01^2 ) + '^1 ('^l'^2 ~ '^2^1 ) = 0, 



flj + ^2>'2 + ^^2 J'l = 02 + ^; 



a,C^ - flj^i 



- + Ct 



a2Ci - 0^2 

bfi2 - CjC2 



bp2-CiC2 " b,b2-CiC2 (9.63b) 

= 02(^1^2 -CiC2)+^2('^l'^2 - ^2^1 ) + ^^2 ('^2*^1 "'^1^2) =0. 

Near-Equilibrium Equation System. The next step is to study the behavior of 
small deviations of the solution from the equilibrium points. Small deviations 
from the equilibrium points are defined by setting 

y,=Y^+v,, y2=Y2+V2. (9.64) 
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Here, (Yy Y2) represent the coordinates of any equilibrium point, and the functions 
(Ui, V2) are small deviations from this equilibrium point. By replacing j, and J2 in 
Eq. (9.60) by the latter expressions we obtain 

^ = (7,+i;J[a,+Z,,(y,+u,)+c,(F,+t;,)]=y,(a,+Z,,F,+c,y,) 
at (9.65a) 

+ u, [qi +biY^ +CiY2 +b^Y^] + V2C^Y^ +u,(Z;,Ui +C1V2), 

^ = fe + t^2 )[«2 + ^2 fe + 1^2 ) + C, (y, + 1^1 )] = ^2 («2 + h2Y2 + C^F, ) ^^ ^^^^ 

+ U, C2F2 +V2 [«2 +^2^2 +'^2^1 +^2^21 + ^2(^2^2 +C2L'l)- 

The first terms on the right-hand sides of these relations are zero because Fi and F2 
are equilibrium solutions. The last terms are quadratic in u, and V2- The latter 
terms can be neglected because Vi and V2 are assumed to be small. In this way, we 
obtain a linear equation system for v 1 and V2, 

— - = Vi[ai +fe,F, +c,F2 +^/i] + 1^2^/1, (9.66a) 

dt 

— - = u, C2F2 + V2 [^2 + ^2^2 + ^2^ + ^2^2]- (9.66b) 

dt 

This linear equation system can be solved in terms of the solutions provided in 
Sect. 9.2.1. 

Generalization. The linearization of the nonlinear equation system described 
in the preceding paragraph can be applied to any nonlinear equation system. The 
latter fact can be demonstrated by considering the nonlinear system 

^ = F,(y„y2), (9.67a) 

dt 

^ = ^2(^1,^2), (9.67b) 

dt 

where Ff and F2 can be any functions of ji and>'2. The Taylor expansion of T^i and 
F2 at an equilibrium point (F,, F2) is then given by 

^ = 7^,(F„F2) + f^(F„F2)U-F,)+fi(F„F2)(j2-F2), (9.68a) 

dt oy-i oy2 

^ = F2(Y„Y2) + ^(Y,My^-Y,)+^(Y^J2){y2-Y2), (9.68b) 

dt oy^ qv2 

where nonlinear powers of ji - Fj and ^2 ~ ^2 ^re neglected (which is justified for 
sufficiently small deviations from the equilibrium point). The functions T*', and F2 
are equal to zero at the equilibrium points, i.e., we have Fi(Yi, Fj) = ^2(^1) ^2) " 0- 
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Hence, we obtain a linear equation system in j, - 7, and j2 ~ ^: 



2) 



dt 



yi-Yi 



dF, , dFy 

. v^'^2) ^lJ'p-f2J / 
oyx oy. 



dF dF 

^iY„Y,) ^{Y„Y,) 



J2 - ^2 



(9.69) 



By calculating the partial derivatives involved, this equation system can be used to 
recover the equation system (9.66), where ^i = Ji - Y^ and ^2=^2" ^2- 

Application. Examples for the application of this approach will be discussed in 
Sect. 9.3. These examples illustrate the benefits of such linear stability analyses. 
However, such analyses are not always successful. The latter is the case if the 
linear system that characterizes the system behavior in the neighborhood of an 
equilibrium point has two pure imaginary eigenvalues such that the trajectories are 
closed curves (ellipses). In that case, small disturbances given by nonhnear terms 
generate positive or negative real parts of the complex eigenvalues. Depending on 
the sign of these real parts, the nonlinear system may be asymptotially stable or 
unstable. Therefore, the analysis of the corresponding linear system does not allow 
in this case to decide whether or not the nonlinear system is asymptotially stable. 
A procedure for handling this problem will be discussed in Sect. 9.4 where 
Liapunov's second method is explained. 



9.3 Population Ecology: Species Interactions 

As a first application of the mathematical concepts presented in Sect. 9.2, let us 
consider the modeling of the multivariate evolution of several populations. This 
problem will be addressed such that the concepts presented for a single population 
in Chap. 7 are extended by the consideration of the interaction of several species. 
The discussions in Chap. 7 showed that there is no unique law of population ecol- 
ogy, but (depending on the definition of the population density function) there are 
many possibilities for formulating equations for population dynamics. In the fol- 
lowing, we will consider modeling approaches that extend the logistic growth 
model for a single population. 



9.3.1 Multivariate Population Dynamics Equations 

Multivariate Evolution. The following discussion of some basic features of 
the multivariate evolution of populations will be based on the nonlinear equation 
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system (9.60), which was analyzed mathematically in Sect. 9.2.3, 



dy_ 
dt 



-p- = J, (fli + b^y^ + c^y, ), (9.70a) 

dt 



yi («2 + hVi + Cz Ji )■ (9.70b) 

dt 

The structure of this model enables the consideration of a variety of types of 
species interactions. We will discuss two examples in the following. 

Competition for Food. As a first example, let us consider the competition for 
food by two species that do not prey on each other. An example is given by two 
species of fish (bluegill and redear) in a pond. The equations considered for this 
case are given by 

^ = yi\a,\-\\\y-\c,\y,\ (9.71a) 

dt 

^ = y^[\a,\-\h,\y,-\c,\yy (9.71b) 

dt 

For the case that there is no interaction between species, i.e., Ci = Cj = 0, these 
equations represent logistic growth models for jvi andjz. The interaction terms that 
involve Cy and Cj do appear here with negative coefficients. In this way, we model 
the food reduction for one species due to the food consumption of the other spe- 
cies. This model will be analyzed in Sect. 9.3.2. 

Predator-Prey Interactions. As a second example, we consider predator-prey 
interactions (e.g., foxes and rabbits in a closed forest). We assume thatjVi refers to 
the prey, and j2 refers to the predator. The equations for this case are given by 

^ = yMaA-\bAy-\cAy,\ {9.ii^) 

dt 

^ = j,(-|a,| + |c,|j,). (9.72b) 

dt 

The prey equation (9.72a) has the same structure as Eq. (9.71a): we have a logistic 
model with an interaction term that is proportional to Cy. A nonzero c, accounts for 
the reduction of prey due to predators. On the other hand, the predator equation 
(9.72b) differs from Eq. (9.71b). The predator will die out in the absence of the 
prey. The consideration of a self-limiting factor (i.e., a nonzero h^ does not make 
sense in this scenario. For a nonzero coefficient C2, the positive last term describes 
the increase of the predator population due to the consumption of prey. Equations 
(9.72) represent the famous Lotka-Volterra equations, which are extended here by 
the consideration of the self-limiting contribution related to the use of a nonzero 
by. This model will be analyzed in Sect. 9.3.3. 
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9.3.2 Competition for Food 

Model Considered. For an analysis of the equation system (9.71) it is helpful 
to consider more specific equations. The parameters hi and foj normalize the other 
model parameters: see Eq. (9.70). Thus, we may set fci = ^2 " ~1 (the negative sign 
is considered according to Eq. (9.71)). The equilibrium values (9.62) show that the 
second and third equilibrium values of Yi and Ft ^re given then by flj and 02, 
respectively. We will assume that a, = Aj ^ 1: which corresponds to the consid- 
eration of normalized population values. Regarding Cj and C2 we assume that Ci = 
Ct = -d, where disa non-negative number. The model considered is then given by 



dyj 

dt 

dyi 



■yXl-yi-dy^l (9.73a) 



= y2{\-y2-dy,). (9.73b) 

dt 

The equations look similar, which leads to the question regarding the difference 
between them. To address this question we calculate the ratio between jvj ^ndj,, 

d J, 1 dy^ y-, dy^ y^ , j \ yi U j \ 

:t-^ = — ±-^-7- = ^^-yi-'^y^)-iyxV-yx-<iy2) 

dt y, y, dt y, at y, y, 

^ ' (9.74) 

=(i-^)^u-j2)=(i-4i-- 

yx \ yx 



yi- 



This relation shows that jj/ji is constant (in particular, we have jj^Ji ^yiol yxo) 
under two conditions: for o? = 1 and for the case that ji andj'T have the same initial 
condition. The conclusion for the latter case can be seen, for example, by writing 
Eq. (9.74) in a discrete formulation. Such a representation shows that there is 
never a change of jz/ji- The latter two cases will be considered first because they 
allow analytical solutions of the nonlinear equation system (9.73). 

Equal Initial Values. For the case j'oo " Jio^ Eq. (9.74) implies jj/ji " j2o/yio 
= 1, which means >'2=ji. According to Eq. (9.73a), ji is then determined by 

^ = j.(l-(l + J)y,). (9.75) 

dt 

This equation is a logistic equation. In particular, this equation corresponds to the 
logistic equation (7.88) by setting Z = 0, r= l,andir= 1 /(I +t/). According to the 
solution (7.101) of the logistic equation, the solution of Eq. (9.75) is given by 

y.- , """'" , . (9.76) 

'^ \i{i+dy 
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The equilibrium solution of Eq. (9.76) is given by Y^ 
j2 =>"!, the equilibrium point for this case is given by 



iY„Y2)- 



1 



1 



l + d l + d 



1/(1 + d). According to 



(9.77) 



Equal Competition. We also have a proportionality jj/ J' i ^ yio^ yio for o? = 1 
where we have an equal competition (because the parenthesis tenns in Eqs. (9.73) 
are equal). According to Eq. (9.73a), j, is then determined by the equation 



dyi 

dt 



■yi 



f 

1- 


^1+^-1 




V 


L ^'loj 


J 



(9.78) 



The logistic equation (7.88) corresponds to the latter equation if L = 0, t= 1, and 
K^ 1/(1 +720 ''jio)- By using the solution (7.101) of the logistic equation, we find 
the solution of Eq. (9.78) to be given by 



Ji 



l/(l + >'2o/j',o) 



1- 



1/(1 + 720 /Jio) 



(9.79) 



The equilibrium solution that is implied by this expression is Fj = 1 /(I +>'2o/>'io)- 
According to j'j^JVi yi^lym, the equilibrium point for this case is 



{"Y.^^i)- 



1 



^20/^10 

l + >'2o/>'lo'l + ;'2o/>^10 



^ f 



V20 



;^io + >'2o >'io + ;'2o 



(9.80) 



Linear Stability Analysis. We have to use linear stability analysis to study the 
behavior of the nonlinear equation system (9.73) for the cases of unequal initial 
values and d+\. From Eqs. (9.62), the equilibrium values of this model are 

(5^,72) = (0,0), (F„y2) = (0,l), (7„72) = (l,0), 



(^1,^2)^ 



\-d \-d 



-d^'\- 



1 



1 



\ + d \ + d 



(9.81) 



Therefore, there are four potential equilibrium states: both species will disappear, 
only one of the species will survive, or there is a coexistence of both species. For a 
non-negative d we find that the Yy and Y^ values are bounded by zero and one, this 
means < 7, < 1 and < Fj - 1- In the following, we will analyze the solution 
behavior in the vicinity of the four equilibrium points (9.81) by making use of the 
linear stability analysis approach presented in Sect. 9.2. 
• (^b ^2) " (0, 0): the linear equation system (9.69) reads for this case 



d[y^-Y, 



dt 1 jj - ^2 



1 OY;v 

iJU 



(9.82) 
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According to Eqs. (9. 12), we have for this case the eigenvalues 

Tj =1, r^= 1, 

Thus, the solution is unstable at the equilibrium point (7i, Fj) " (0, 0). 
• (7i, Fj) " (0, 1): the linear equation system (9.69) reads for this case 



d_ 
dt 



yi 



\-d 

-d 



": 



The eigenvalues are given by 



d_ 

~1 



(9.83) 



(9.84) 



(9.85) 



For the case 1 - dl 2>Q, the eigenvalues are given hy i\ = \ - d and r, = -1. For 
the case 1 - o?/ 2 < 0, the eigenvalues are given by /'i = -1 and r2 ^ 1 - d. It is up 
to us which eigenvalue we called /', and i^- Thus, we can use 

(9.86) 



r,=l- 



r,=-l. 



Depending on the value of d, the eigenvalue }\ can be negative or positive. Thus, 
the solution can be asymptotically stable or unstable at this equilibrium point. 
• (Fi, Fj) " (1, 0): the linear equation system (9.69) reads for this case 



d_ 
dt 



y\ 



■1 



Ji 



The eigenvalues are given by 

1-- 



d 



(9.87) 



(9.88) 



That are the same eigenvalues as found for (F,, Fj) = (0, 1). Correspondingly, the 
eigenvalues are again given by 

}\=\-d, r2=-l, (9.89) 

i.e., the system behavior is the same as in the vicinity of (F,, Fj) = (0, 1). 

• (^i> ^2) = (^) ^) where F= 1/(1 + d): the equation system (9.69) now reads 



d_ 
dt 



y^-^x 
yi-Yi 



\-{2 + d)Y -dY 

-dY l-{2 + d)Y 



yi 
yi 



'27 



The eigenvalues are given by 
r, =l-(2 + ^)F + ^F = l-2F: 



\ + d-2 



-\ + d 



\ + d \ + d 

r,=\-(2 + d)Y-dY = \-2{\ + d)Y = \-2 = -\. 



(9.90) 



(9.91) 
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Fig. 9.3. The competition of two species j'l and jj for food. The solutions ji and 3/2 
of the equation system (9.73) are shown in (a), (b), and (c) as function of time t for 
d = (0.5, 1, 1.5), respectively. The initial conditions j^io = 0.2 and jjo " 0-1 are 
used. The phase plane evolution of j, and J2 is shown for several initial conditions 
in (d), (e), and (f) for the cases d = (0.5, 1, 1.5), respectively. The dashed line in 
(e) is not a realizable trajectory: is only gives an orientation regarding the equilib- 
rium points for this case. 



As given for the equilibrium point {Yy, 7,) = (0, 1), the eigenvalue Vy can be nega- 
tive or positive depending on the value of d. Consequently, the solution can be 
asymptotically stable or unstable at this equilibrium point. 

Illustration. We will assume that there are two populations initially such that 
the initial values jio and J20 are nonzero. Then, the equilibrium point (0, 0), which 
is characterized by two positive eigenvalues, can never be realized. Hence, it is 
impossible that both populations disappear. For equal initial values jjo " Jio we 
find the solution (9.76) for j, and we have y^ = y^. The equilibrium solution is 
(^i5 ^2) = (1 / (1 + <i), 1 / (1 + <i)). For unequal initial values we find features that 
are illustrated in Fig. 9.3 for d = (0.5, 1, 1.5) and several initial conditions. For a 
relatively weak competition {d < 1), we find the development of a coexistence 
(^1) ^2) = (1 /(I + <i), 1 /(I + ^) between both species. This result agrees with the 
conclusions of linear stability analysis: the equilibrium points (0, 1) and (1, 0) are 
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characterized by one positive eigenvalue, whereas the coexistence point (Y, Y) is 
characterized by two negative eigenvalues. The equal competition case with d ^ 1 
does still allow a coexistence of species, but the initial values matter in this case: 
the species with the higher initial value will have a higher equilibrium value (see 
Fig. 9.3b). The solution for ji is given for this case by the logistic function (9.79), 
and we have yj = Ji J20 ^ Jio- The equilibrium point is given by Eq. (9.80). For a 
relatively strong competition (d > 1) we find that one species disappears whereas 
the other species achieves a maximum value. In particular, we find that the species 
with the higher initial value will survive. This observation is also supported by 
linear stability analysis: the coexistence point (Y, Y) has one positive eigenvalue, 
and the equilibrium points (0, 1) and (1,0) have two negative eigenvalues. 



9.3.3 Predator-Prey Interaction 

Model Considered. The predator-prey equations (9.72) will be also analyzed 
by considering more specific equations. In correspondence to Eq. (9.73) we apply 
a, = 1 and 02 = -1. We assume an equal amount of interaction by setting Ci = -4 
and C2 = 4. In addition, we assume that hi = -e, where e is a non-negative param- 
eter. With these assumptions, the equation system considered is given by 

^ = j,(l-ej,-4j,), (9.92a) 

dt 

^ = j,(-l + 4j,). (9.92b) 

dt 

The specific relevance of e variations can be seen by considering 

dy2 dy2ldt y-, -1 + 4V| 



4Vi dyjdt J, 1-ej, -4^^ 



(9.93) 



This equation is a separable equation for e = 0, which means that this equation can 
be solved. 

Zero Self-Limitation. First, let us find the analytical solution to the nonlinear 
equation system (9.92) for the case e = 0. Relation (9.93) can be written then 

i^^^,,-^il^^,,=0. (9.94) 

yi yx 

The integration of both sides provides 

ln|3;,|-4j,+ln|jj-4j, =C. (9.95) 
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Here, C is a constant that is determined through the initial conditions (by setting 
? = on the left-hand side). Unfortunately, it is impossible to use this relation for 
the calculation of jj as an explicit function of yi. The relevance of Eq. (9.95) is 
that this relation describes a closed curve (see, for example, the illustration of this 
case in Fig. 9.4d). The existence of a closed curve means that we have a stable 
solution for e = 0. Hence, nonzero e values describe deviations from a stable state. 
Interestingly, a closed curve is also found for any other parameter values than 
those used in Eq. (9.92), provided these parameters have the same signs and e = 0. 
Linear Stability Analysis. We have to use again linear stability analysis to 
understand the behavior of the nonlinear equation system (9.92) for the case eitQ, 
According to Eq. (9.62), there are three potential equilibrium points for this sys- 
tem (the equilibrium point (0, -Oj/ftj) in Eq. (9.62) cannot be realized), 

(9.96) 
Correspondingly, it is possible that both species will be extinct, or only the prey 
survives, or there is a coexistence of both species. The equilibrium values Yi and 
Y2 are positive if e < 4. The setting e = 4 does recover the second equilibrium point 
(1/4, 0). For simplicity, we do not consider this case e = 4, this means we 
consider variations < e < 4. The solution behavior of Eqs. (9.92) in the vicinity 
of the three equilibrium points (F,, Fj) reveals the following features: 
• (Fi, Fj) = (0, 0): the linear equation system (9.69) reads for this case 



dt[y2-Yj [0 -lJU-^2 



(9.97) 



According to Eqs. (9.12), the eigenvalues are given by 

'1=1, r, =-1, (9.98) 

Correspondingly, the solution is unstable at the equilibrium point (F;, Fj) = (0, 0). 
• (Fi, Fj) = (1 /e, 0): the linear equation system (9.69) reads for this case 



(9.99) 



-l + --- = -l. (9.100) 

e e 

The eigenvalue r, is positive due to the condition < e < 4. Thus, the solution is 
unstable at the equilibrium point (F,, F2) = (1 /e, 0). 
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[y.-YA 


'-, -11 
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(yx- 


-Yy 


dt 
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(7i, Fj) = (1/4, [1 - e/4] /4): the equation system (9.69) now reads 



d_ 
dt 



yi 



'2j 



1 - 2ey, - 472 
4K 



-47, 
-1 + 47 



yi 



(9.101) 



'2j 



By adopting the definitions of 7, and Y2 the latter equation system can be written 
in the following way, 



d_ 
dt 



y\ 
yi 



1- 



-1 



yi- 



e 


-1 


/ 


4 




i^i 


e 





Ij2 


4 


/ 





■7, 



(9.102) 



Therefore, the eigenvalues are given by 



e 1 

-— + — 

8 2 



16 



■4 1 



e' 
16 



■4 1 




(9.103) 



Depending on the e variation, < e/4 < 1, there are three cases of eigenvalues. A 
first case is given for e = 0, which means that we have two complex eigenvalues 
with zero real parts. As discussed above, this case corresponds to a stable solution. 
For a nonzero e, we have to distinguish cases for which the square root is real and 
imaginary. The square root becomes zero for e/4 = ± 8"* - 2. Due to the variation 
< e / 4 < 1 considered, only the value e/4 = 8"^ -1 = 0.8284 can be realized. 
Correspondingly, we may have two cases in addition to the case e = 0. For the 
case < e/4 < 0.8284, we have two complex eigenvalues with negative real part. 
Thus, the solution is asymptotically stable. For the case 0.8284 < e/4 < 1 we have 
a real square root. The eigenvalues r, and Tj are always negative for this case. 
Hence, the solution is again asymptotically stable. 

Illustration. An illustration of solutions of the equation system (9.92) is given 
in Fig. 9.4 for the cases e/4 = (0, 0.6, 0.9). We consider nonzero initial popula- 
tions densities y^f^ and jio- This assumption implies that the equilibrium solutions 
(0, 0) and ( 1 / e, 0), which are both characterized by one positive eigenvalue, can 
never be realized. Consequently, no population will disappear, which means that 
there will be a coexistence between both populations. For a zero self-limitation 
(Fig. 9.4a, d), we observe cyclic variations of the predator and prey populations: a 
decrease (increase) of prey leads after a delay time to an increase (decrease) of 
predators. An equilibrium state cannot be established in this way. In the phase 
plane, the latter behavior corresponds to a closed curve that surrounds the center. 
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Fig. 9.4. Interactions between a predator jj and prey ji. The solutions >>! and j2 of 
the equations (9.92) are shown in (a), (b), and (c) as function of time t for the 
cases e / 4 = (0, 0.6, 0.9), respectively, where jio = J20 " 0.5. The corresponding 
phase plane evolution of ji and j'2 is shown for several initial conditions in (d), (e), 
and (f), where e / 4 = (0, 0.6, 0.9), respectively. The dots show the equilibrium 
points. 



The yi-yj curves follow Eq. (9.95), and the center is located at (1/4, 1/4), see 
Eq. (9.96). For a nonzero but relatively weak self-limitation < e / 4 < 0.8284 
(see Fig. 9.4b, e), the prey curve shows oscillations that are damped out due to the 
self-limitation (which appears as a sink term in Eq. (9.92a)). Due to the coupling 
with ji, the predator curve also shows damped oscillations. The damping implies 
in the phase plane an asymptotically stable solution. For the case of a relatively 
strong self-limitation 0.8284 <e/ 4 < 1 (see Fig. 9.4c, f), the damping does not 
allow oscillations anymore. After the first minimum (maximum), jj (yj) realizes 
the equilibrium value. It is interesting to see that the increasing damping reduces 
the Y2 coordinate of the coexistence point (7,, Fj) = (1/4, [1 - e/4]/4), whereas 
the Yi coordinate is unaffected. The latter fact is a consequence of Eq. (9.92b), 
which fixes the stationary value Yi = 1/4. A modification of this equation (e.g., by 
the addition of a positive term proportional to J2 in the parenthesis term) would 
lead to different features. 
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Fg = mg Fig. 9.5. An illustration of a pendulum. 



9,4 Mechanical Motions: The Pendulum 

Let us consider next evolution principles for vector processes in mechanics. In 
continuation of the explanation of laws for mechanical processes in Chap. 7 we 
will focus the discussion in this chapter on the application of Newton's Laws of 
Motion. In particular, we will extend the discussion of one-dimensional harmonic 
oscillator motions in Chap. 7 by considering now the motions of a pendulum. The 
results obtained in this way were used in Chap. 3 regarding the discussion of the 
measurement of time. 



9.4.1 Pendulum Equations 



Newton's Laws of Motion. Contrary to population ecology we have a sound 
mathematical basis for the modeling of mechanical processes given by Newton's 
Laws of Motion, which were discussed in Chap. 7. Mechanical motions of macro- 
scopic bodies that move with velocities much smaller than the speed of light can 
be described by Newton's Second Law given by Eq. (7.33), 






m 



(9.104) 



Here, x = (xi, Xj, Xj) is the position vector of any body, F = (F^, F2, F^) is the force 
acting on the body, and m is the mass of the body. The use of this equation for the 
calculation of pendulum motions will be demonstrated in the following. 

Undamped Pendulum Equation. An illustration of the pendulum considered 
is given in Fig. 9.5. A mass m is attached to one end of a rigid, but weightless, 
supported rod of length r. The rod is free to rotate in one plane. The angle a{t) is 
the angle of displacement from the vertical. The force that drives the pendulum is 
given by the gravity force F^^mg. Here, g denotes the gravity acceleration. Other 
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forces are not involved regarding the undamped pendulum motion. The x-y coor- 
dinate system applied is shown in Fig. 9.5. In correspondence to the analysis of 
the spring-mass system we assume that the downward direction y is the positive 
direction. The equations that govern the motion of the pendulum (the changes of 
the x(t) and y(t) coordinates of the pendulum) are given by Newton's Second Law 
(9.104). 

d X 
dt 



2 -0, (9.105a) 



4^ = ^ = g. (9.105b) 

dt m 

To take advantage of the fact that the pendulum moves along a circle with con- 
stant radius r, it is helpful to switch to polar coordinates given by the radius r and 
the angle of displacement a. According to the illustration in Fig. 9.5, the relations 
that relate (x, j) and (r, a) are given by 

X V 

sin« = — , cos« = — . (9.106) 

r r 

The first-order and second-order derivatives of x(?) and y({) that are imphed by 
these relations are given by (r is constant) 

dx da d^x . ( da\ d^a ^„1/^-, n 

— = rcosa , — r- = -rsm« +rcos« — —, (9.107a) 

dt dt dt \dt ) dt- 

dy _ . da d^y _ ( da\ . d^a 



-rsino; , — ^ = -rcos« -rsin« — — . (9.107b) 

dt dt dt \dt ) dt 

The use of these relations in Newton's Second Law equations (9. 105) then implies 



(da 
rsin« 



rcos«^ = 0, (9.108a) 

dt 



( da\ . d a 
-rsin« — r 

dt ) dt 



rcos« — ;— I -rsina , ^ = g- (9.108b) 



Equation (9.108a) can be used to replace the quadratic first-order derivative by the 
second-order derivative of a. 



da V cosa d a 



dt ) sin« dt' 
The use of this relation in Eq. (9. 108b) leads then to 



(9.109) 



cos« 

cos« + sm« 

sin« 



d^a cos'« + sin^« (i*« 1 d^a 



, , ^, (9.110) 

dt' sin« dt sm« dt r 
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where the Pythagorean identity was applied. Hence, the equation of motion for the 
undamped pendulum reads 

^ = -^sina. (9.111) 

dr r 

Damped Pendulum Equation. In general, the pendulum will be also affected 
by a damping force, which reduces the pendulum velocity due to the air resistance. 
In correspondence to the analysis of the spring-mass system we assume that this 
damping force is proportional to the pendulum velocity dal dt. Hence, we extend 
Eq. (9. 1 1 1) in the following way, 

d'a I da g . zn 1 n^ 

— r- = — sin«. (9.112) 

dt T dt r 

For the characteristic damping time scale we use Stokes' Law (see Sect. 3.3.3), 

m 

(9.113) 



6n /ui'p 

Here, // refers to the dynamic viscosity, and ;> is the radius of the spherical mass. 
The damping contribution appears in Eq. (9.1 12) with a negative sign because this 
term reduces the pendulum velocity dal dt (the damping term implies that dal dt 
becomes smaller for a positive dal dt). Similar as in the discussion of damping in 
the spring-mass system, the structure of the damping term applied here does only 
represent one reasonable assumption among several possible choices. The pendu- 
lum equation that results from the use of Eq. (9. 113) then reads 

d a dn uvp da g . ^ /„ , , .x 

— Y + ^—^ + — sin«=0. (9.114) 

dt m dt r 

Normalized Damped Pendulum Equation. The use of nondimensional varia- 
bles is helpful because the number of model parameters involved in the equation 
can be reduced. We introduce the nondimensional time t,^tl (r /g)"", such that 
Eq. (9.114) reads 

d'a bTi/urp \r da . _ /n11c^ 

— - + ^— -^ + sma = 0. (9.115) 

dt, m \g dt. 

By introducing the nondimensional dynamic viscosity //, = // {r" I gY^ I m, which 
can be seen as an inverse Reynolds number, this equation can be written 

d^a , Vp da . ^ zn i i/c\ 

— ^ + 6;r//.-^ — + sma = 0. (9.116) 

dt, r dt. 

We introduce the nondimensional variable d = d nvpl r to simplify the writing of 
this equation. Then, the damped pendulum equation, which does now only depend 
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on one parameter (the product d/j,), is given by 

d^a , da . ^ m i n\ 

— :r + diu, + sin«=0. (9.117) 

dt, dt. 

Nonlinear Equation System. The second-order differential equation (9.117) 
can also be represented as an equation system. We set ji = a and Jt ^ da/ dt,. The 
differentiation of ji and j2 provides then 

§^ = ^ = 7. (9.118a) 

dy2 d^a , da . , . zn 1 1 ou\ 

— - = — r- = -d /u, sm« = -d /u,y2 -smjj. (9. Hob) 

dt, dt, dt. 

The initial values for j, and jj are given by jio = «(0) and J20 " da/dt,{0). 



9.4.2 Linear Stability Analysis 

Let us analyze the nonlinear equation system (9.118) by adopting the linear 
stability analysis approach described in Sect. 9.2.3. 

Equilibrium Points. First, we have to determine the equilibrium solutions of 
Eq. (9.1 18). Such equilibrium solutions have to satisfy the equations 

Q = y„ (9.119a) 

Q = -d/u,y,-?,my,. (9.119b) 

These two equations are solved by j^i = ± n 71 and y 2 = 0, where « = 0, 1, 2, .... 
However, there is no need to consider all these equilibrium points. We do only 
have to consider the two physical equilibrium solutions (0, 0) and {n, 0). Due to 
physical reasons we expect that the first equilibrium point (0, 0) is asymptotically 
stable and the second equilibrium point {k, 0) is unstable. 

First Equilibrium Point: Linear Stability Analysis. According to Eq. (9.69), 
the linear equation system that describes the pendulum motion close to the first 
equilibrium point (0, 0) reads 



d_ 
dt. 



^jiWo 1 ^{y^ 



„ , (9.120) 

jj \-l -d/u,)yy2^ 



By differentiating dyj dt, = jj ^^d using dy2l dt, = -ji -d n,y2, we find in terms of 
the original variables y\ = a and y2 = da I dt, the equation 

d^a J da m ni\ 

— :r = -a-du, . (9.121) 

dt,^ dt. 
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Hence, the analysis of this case corresponds to the consideration of the linearized 
pendulum equation (9.117) where sin a is approximated by a, which is justified 
for sufficiently small initial angles of displacement. The equation system (9.120) 
represents a specific case of the linear equation system (9.2). The eigenvalues of 
Eq. (9.120) are, therefore, given by Eq. (9.12), 

''i='"s+'d. r^^r^-r^, (9.122) 

where r^ and r^ are given by 

I's =-diu,l 2, r^ = ^{d/jjlf-l. (9. 123) 

The effect of damping is relatively small in general. Therefore, we may assume 
that djU, < 2. For this case we find r^ = / r^,., where the real number r^,. is 

r^,=^l-idMJ2)\ (9.124) 

Both eigenvalues have a negative real part r^ if there is a nonzero damping, i.e., 
d iJ,^Q. The discussion at the end of Sect. 9.2.2 showed that such a system is 
asymptotically stable. The solution to the linear equation system (9.120) can be 
found by making use of the fact that Eq. (9.121) represents a specific case of the 
homogeneous linear second-order differential equation (7.45). According to Eq. 
(7.69), the solution of Eq. (9.121) is then given by 



, , .,<. sin(r^,?.) 
a{t,) = e \ a\ 



sin(ra.?,) 
cos(r^.;.) r^ 



(9.125) 



■■ «o e' '' \ zo%{rj^, t, ) + — ° ^^ ° sin(rg. t, ) 



Here, «o refers to the initial angle of displacement «(0), and a\ is the initial value 
of dal dt,. The latter relation can be rewritten by defining an angle Shy 

i3.n5= "''~'''"\ (9.126) 

The use of this relation then enables the following rewriting of Eq. (9.125), 

a{t,) - [cos(rQ,^,) + sin(rQ.?,)tan jje'^'' a^ 

cos(r^,?,)cos(? + sin(r£,.?,)sin(J ,^ cos{rij,t, - S) ,, (9.127) 

= z e «o = J e «o- 

coso coso 

By replacing r^ and r^, according to their definitions (9. 123) and (9. 124) we find 



cos ^1 - (d ju, 12)' t,-5 



a{u) = ^ '-e"^"''^a„ (9.128) 

cosd 
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where the angle Sis given by 
S = arctan 



«'(,/«5 +d juj2 



yjl-(djuJ2y 



(9.129) 



The solution of the equations (9.120) is then given hy y^ = a and>'2 = dal dt,. 

Second Equilibrium Point: Linear Stability Analysis. At the second equilib- 
rium point {k, 0) Eqs. (9.69) imply the linear equation system 



A. 

dt. 



y-i 



1 

■d 1^, 



yi 



(9.130) 



Here, dy^l dt, was replaced by d(yi - k)I dt,. To solve this equation system we use 
the equivalent second-order equation 



d'ia-Ti) , dia-n) 

= a-n-d 1^1,- 



dt. 



dt. 



(9.131) 



where yx = a and Jt ^ d.O-1 dt, are used. The latter equation can be obtained in the 
same way as Eq. (9.121). The eigenvalues can be written 



-djU./2 + -^l + {djU,/2y, 



-dju,/2-^j\ + {dju,/2y. (9.132) 



The eigenvalue r^ is always positive (also for zero damping), i.e., the solution near 
the second equihbrium point is unstable. According to Eq. (7.57), the solution of 
Eq. (9. 13 1) is given for this case of two unequal real eigenvalues by 



«'„-r, («„ -;r) r.t, a\.-rAa„- n) r.t. 
a -71 = — - — - — e — ■ — e' 



(9.133) 



The solutions of Eqs. (9.130) follow then from j, = a and y2 = da/ dt,. Regarding 
the discussion of the evolution in the y[-y2 phase plane below it is interesting to 
consider the consequences of setting o;'o = '"2 (c(o ~ ^), such that the first term in 
Eq. (9.133) disappears. By differentiating the resulting expression we find 



da 
dt. 



a', 



-r, («o-;r) ,-,,. 



-r^ia-K), 



(9.134) 



which means that j, " ''2 C^i ~ ^) for all t. Similarly, we find for a'a = ri{aa- n) 
da _ _ a\-}\{af^ - k) ,-,,, 



dt. 



}\{a-n). 



(9.135) 



which means that j'j ^ ''i C^i ~ ^) for all t. Hence, the yi-yi phase plane figure will 
involve the two linear functions jj ^ ''2 (Ji ~ ^) and jj ^ ''i CVi ~ ^r) in the vicinity 
of the point {k, 0) for both the damped and undamped pendulum. 
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Fig. 9.6. An illustration regarding the calculation of the 
potential energy of a pendulum. The pendulum is lifted 
above its minimal position by the distance r(l - cos d). 



9.4.3 Nonlinear Stability Analysis 



Lyapunov's Second Method. The use of linear stability analysis leads to very 
helpful conclusions, but the suitability of the assumption of linear processes in the 
vicinity of equilibrium points is often not very clear. A nice way to overcome this 
problem by the analysis of the nonlinear equation system (9.118) was developed 
by Lyapunov. This approach, which will be presented in the following, is known 
as Lyapunov's second method (Lyapunov's first method refers to the method of 
linearization of a nonlinear equation along an orbit). The basic idea of Lyapunov's 
approach is the consideration of orbits (i.e., trajectories in the phase plane) that are 
characterized by decreasing values of a non-negative function (which is called the 
Lyapunov function). The trajectory and its Lyapunov fiinction will change until 
the Lyapunov function reaches the value zero. The position of the trajectory in the 
phase plane at which the Lyapunov function is equal to zero characterizes an equi- 
librium point. Hence, the asymptotical stability of nonlinear equation systems can 
be shown by proving the existence of a Lyapunov function that decreases to zero. 

Pendulum Lyapunov Function. The most natural choice for the pendulum 
Lyapunov function is the total energy E defined by 



E = mgr{\ -coso;) H — mr 

2 \ dt 



(9.136) 



The first contribution represents the potential energy (the work done in lifting the 
pendulum above its minimal position: see the illustration in Fig. 9.6). The second 
contribution is the kinetic energy of the pendulum. To simplify the analysis below 



we use the nondimensional time t, 
sional energy E, ^ El{mgr), 



1 1 {r I g) , and we introduce the nondimen- 



E, = l-cos« + 



da 

dt. 



■ l 



^1 2 
-COSJ1+-J2 , 



(9.137) 
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where the definitions yi = a and j2 ^ da/ du are applied. The total energy E, has 
two relevant properties. The first property is that E is non-negative, 

£, >0. (9.138) 

The case E, = can only appear if jj ^ ^nd yi= 2 n n, where n = 0, ±1, ±2, . . . 
(such that cos ji = 1). Hence, E, ^ for all the asymptotically stable equilibrium 
positions. The second property of ii. is the inequality 



dE, . dy, dy., . / , ■ \ , 2 ^ 

— — = smj,— -L + J2— ^^jjSinji -j2(t///.j2 +smj,j=-t///,j2 < 0. 
dt, dt, dt, 

(9.139) 

The derivatives of ji and jj are replaced here according to Eqs. (9.1 18). 

Nonlinear Stability Analysis. By excluding equilibrium points as initial points 

we find, therefore, the following results of this discussion. 

• The damped pendulum motion is characterized by a decreasing energy E, except 
for the case that yj = 0. There are two possibilities to find this case. First, we 
have J2 " at the equilibrium points y^^ + n 71 with « = 0, ± 1 , ±2, .... However, 
the equilibrium points can only be realized asymptotically. Second, jj ^ at the 
points on the left and right side at which the pendulum reverses the direction. 
However, E, continues to decrease after passing these points. Thus, the damped 
pendulum motion is characterized by trajectories with dEJ dt <0 that approach 
the asymptotically stable equilibrium points with j'2 ^ and ji = 2 « n:, where n 
= 0, ±1, ±2, . . . (because E, = at these points: see Eq. (9. 137)). Hence, the other 
equilibrium points (as (-;r, 0), (;r, 0), (3;r, 0), . . .) are asymptotically unstable. 

• The undamped pendulum motion {d jj, = 0) is characterized by a constant value 
E,> (asymptotically stable equilibrium positions are not considered as initial 
points). The curves satisfy the equation 

1-cosj, +->'2' =£,. (9.140) 

The type of curve depends on the value of i?.. For relatively small E, values we 
have relatively small ji and 3^2- By approximating cosji by its Taylor series at 
(0, 0) in the first order of approximation, cosy^ = 1 -yi^/2, Eq. (9.140) becomes 

y^'+y,'=2E,. (9.141) 

This equation describes a circle centered at (0, 0) with radius (2E,y'^. For larger 
Et values we have to consider the curve formula 



J2 



±^2(£. +COSJ1-I), (9. 142) 



which is implied by Eq. (9.140). Closed curves correspond to stable cychc 
motions about the equilibrium point. Closed curves must include the possibihty 
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that 3^2 ^ da / dt, = such that the pendulum can reverse the direction. The 
function (9.142) shows that the value y2 = Q can be realized as long as ii. < 2 
(otherwise E, is always larger than cos ji -1). Therefore, open curves (corre- 
sponding to values Ji = « that do always increase: see the illustration in the next 
subsection) are found if E, > 2. The case ii, = 2 is a specific case that separates 
closed and open curves. For this case we have 

y^ = ±^2(2 + cosy ^-\). (9. 143) 

This curve is called the separatrix for the undamped pendulum motion. 



9.4.4 Pendulum Motions 

Nonlinear pendulum motions are illustrated in Fig. 9.7 as functions of time and 
in the yry2 phase plane. Typical features of these motions will be discussed next 
and compared to the conclusions of stability theory. 

Undamped Pendulum. The undamped pendulum is characterized by two sorts 
of areas: areas that are bounded from below and above by separatrices (indicated 
by the differently shaded areas in Fig. 9.7c), and the remaining areas. The shaded 
areas are characterized by closed curves in the y\-y2 phase plane corresponding to 
cyclic pendulum motions: see the ji(0 curve forjjo" ^/3. The system behavior is 
very different outside the shaded areas: we have here open curves in the y^yi 
phase plane that are related to a steady increase ofji = a: see the ji curves related 
tojjo = 2 ;r/3 andj^jo = 3 ;r/2. The separatrices separate these two behaviors. The 
separatrices are closed, but the corresponding yi{t) curve does not show cyclic 
variations anymore: see the curve jVi(0 that results fromj'jo ^ 2"^. 

Comparison with Stability Theory. These observations agree with the conse- 
quences of linear and nonlinear stability theory. The local phase plane features are 
explained by the linear stability theory. The two linear functions J2 ^ ''2 (ji ~ ^) 
and J2 " fi iyi ~ ^) are found in the vicinity of the point (;r, 0): V2 = ''i (yi - ^) is 
the increasing function, andy2 = '"2 (yi - tt) is the decreasing function. The global 
phase plane features are explained by the nonlinear stability theory. The curve 
shapes correspond to the conclusions reported in Sect. 9.4.3 as a consequence of 
analyzing the Lyapunov function E,. For relatively small E, values we find cir- 
cles, the separatrix obtained for ii, = 2 is described by Eq. (9.143), and for large 
values of ii. we find open curves. 

Damped Pendulum. The phase plane for the damped pendulum is differently 
organized. All the space is divided into areas that are enclosed by separatrices: the 
differently shaded areas in Fig. 9.7d are surrounded by other areas that are also 
enclosed by separatrices. The calculation of separatrices is not as simple as for the 
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Fig. 9.7. Pendulum motions. Solutions ji = « of the nonlinear pendulum equation 
system (9.118) are shown as function of time t, in (a) and (b) for the undamped 
(fi?//, = 0) and damped pendulum {d /u, = 0.2), respectively. The curves start at 
jio = 71 1 2. The initial values J20 have the values given in the figures; (c) and (d) 
illustrate undamped and damped pendulum motion, respectively, for a variety of 
initial conditions in the y^yj phase plane. Separatrices are indicated by dashed 
lines. The differently shaded areas indicate areas enclosed by separatrices. 



undamped pendulum because the choice of initial values for the trajectories is not 
obvious. In particular, the lowest separatrix has to be calculated such that it ends 
in (0, 0). This curve is determined by the initial values (3 71, -3.574). The middle 
separatrix between ;rand 3 ;rcan be calculated by the initial vales (5 71, -3.574). 
The other separatrices follow from symmetry conditions: The middle separatrix 
between -71 and k follows from the initial value (-3 71, 3.574), and the highest 
separatrix follows from the initial value (3 71, 3.574). Curves that begin inside the 
lower shaded area in Fig. 9.7d are attracted by the equilibrium point (0, 0), where- 
as curves that begin inside the upper shaded area are attracted by the equilibrium 
point (2 71, 0). Correspondingly, curves that begin in other areas are attracted by 
different asymptotically stable equilibrium points. For example, the lowest solid 
curve in Fig. 9.7d is attracted by (-2 71, 0), and the highest solid line is attracted by 
(4;r,0). 
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Comparison with Stability Theory. These observations do also agree with the 
conclusions of stability theory. According to linear stability theory, the linear 
functions j', = '"2 (Ji ~ ^) ^ndj^j = i\ (yj - n) are found at {n, 0): jj = r, (y, - n) is 
the increasing function, and^j = r, (yi - 71) is the decreasing function. According 
to nonlinear stability analysis, all trajectories approach the asymptotically stable 
equilibrium points withyj " andji = 2 « ;r, where w = 0, ±1, ±2, .... Thus, the 
other equilibrium points (-;r, 0), {71, 0), (3 71, 0), ... are asymptotically unstable. 



9,5 Fluid Dynamics: Lorenz's Weather 

Next, let us analyze Lorenz's weather. The latter term refers to a simple model 
for the explanation of convection, which is relevant to atmospheric motions (the 
motion of the atmosphere is forced by the latitudinal imbalance of solar heating) 
and many technical applications (e.g., the design of heat exchangers). The consid- 
eration of convection here continues the discussion of simple convection models 
in Sect. 7.2. The model equations considered represent a simplification of the 
complicated partial differential equations of fluid dynamics derived in Chap. 10, 
which are implied by the Newtonian mechanics. A main feature of fluid dynamics 
equations is that these nonlinear equations generate deterministic chaos (chaotic 
solutions). It will be shown below that Lorenz's weather model is characterized by 
the same feature. From a methodological point of view, the equations considered 
extend the previous analyses by the consideration of three coupled equations. 



9.5.1 The Lorenz Equations 

Lorenz's Equations. Following the studies of Saltzman (1962), Lorenz (1963) 
suggested the following equations to investigate basic features of convection, 

^ = Pr{y,-y,), (9.144a) 

at 

^ = yAR-y,)-y2, (9.i44b) 

at 

^ = y^y2-by,. (9.i44c) 

at 
These equations represent a highly simplified model for (Rayleigh-Benard) con- 
vection. Lorenz's equations can be also seen as a toy model for weather, which 
can be used for explaining the limitations of long-range weather forecasting (see 
e.g., Gleick 1987, Lorenz 2006, Baines 2008, and Boyce & DiPrima 2009). 
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Fig. 9.8. An illustration of convection according to Lorenz's equations. Fluid flow 
is considered in a single cell (i.e., in one box). The flow is heated from below and 
cooled from above, and there are slippery nonconducting side walls. For a suffi- 
ciently large temperature difference the warmer fluid rises, cools down at the top, 
and moves downwards. This results in a steady convective motion. The strength 
and direction of this circulation is measured by the variable ji. 



Figure 9.8 shows an illustration of the case considered. The model variables have 
the following meaning. The strength and direction of the circulation is measured 
by Ji, yi measures the horizontal temperature variation, and y^ measures the 
vertical temperature variation. The model parameter Pr is the Prandtl number (the 
ratio of diffusivities of momentum and heat), and b is defined hy b ^ 4 / {I + a^). 
The parameter a is a horizontal wavenumber for the convection cells, and b meas- 
ures the width-to-height ratio of the convection layer. The most relevant model 
parameter is R, which is proportional to the vertical temperature difference (the 
driving force of the system). In particular, R is defined hy R^ R^l R„ where R„ 
refers to the Rayleigh number and R^ refers to the critical value of the Rayleigh 
number (the Rayleigh number that is required for the onset of convection). A large 
value ofR implies a large thermal forcing of motion. 

Lorenz Model Considered. To simplify the relatively complicated analysis of 
Lorenz's equations (9.144) we will follow the studies of Saltzman and Lorenz by 
specifying a^ = 1/2 so that b = 8/3. We also specify Pr = 10, which is a reahstic 
value for water. The equation system that results from these assumptions reads 



(9.145a) 
(9.145b) 
(9.145c) 



These equations will be analyzed in the following in dependence orR> 0, which 
controls the amount of thermal forcing. 



dy, 
dt 


= 10(^2 -Ji), 


dy, 
dt 


= yiiR-y^)-y2, 


dy, 
dt 


8 

= yiy2--yi- 
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9.5.2 Linear Stability Analysis 



Equilibrium Points. Linear stability analysis has to be applied to derive ana- 
lytical conclusions regarding the nonlinear equation system (9.145). Such analysis 
requires the calculation of equilibrium points. These points are defined by 



= miy,-y,), 

= Jl(-^-J3)->^2' 



0: 



■yiyi 



;y3- 



(9.146a) 
(9.146b) 

(9.146c) 



Equation (9.146a) implies jVj ^Ji- Hence, the other two conditions can be written 

= y,{R-y,-^\ (9.147a) 



0: 



■yi 



rJ3- 



(9.147b) 



The first way to satisfy Eq. (9. 147a) is given by y^ = 0, which implies Ji ^ J3 ^ 0- 
The second possibility to satisfy Eq. (9.147a) is y^ ^ R - 1. This setting implies 
that j2 = Ji = [8 (R - l)/3]'", or J2 = Ji = "[8 (R - l)/3]"l Hence, we have three 
equilibrium points given by 



P, = (0, 0, 0), P, = (V8(i?-l)/3, V8(^-l)/3, R - 1), 

P3=(-V8(^-l)/3,-V8(7?-l)/3,7?-l). 



(9.148) 



The properties of these equilibrium points depend on R. For R < I, the only real 
equilibrium point is given by Pi. For R> I, there are three real equilibrium points. 
For 7? = 1 we have three times the equilibrium point (0, 0, 0). 

Linear Stability Analysis. The linear equation system in the neighborhood of 
any equilibrium point can be obtained by generalizing the equation system (9.69) 
to the three-dimensional case. 



d_ 
dt 



r 


-^0 


f 


yi 


-5^2 


= 


Us 


-^^ 


V 



^F^/^y^ dF^ldy^ 
dFjIdy^ dFj^ldy-^ 
dFJdy^ dF^ldy^ dFJdy 



dF,/dy,^ 
dF^/dy^ 



3V,,,=y,V' 



yi 

yy 



(9.149) 



'3 J 



Here, F^, F2, and F^ represent the right-hand sides of the three equations (9.145), 
respectively. Regarding Eqs. (9.145) considered we find 



d_ 
dt 



fy.-YA 


f 


72-^2 


= 


[y^-Yj 


V 



10 10 







R-Y, 



■1 



i/2 



yi 
yi 

y^- 



(9.150) 
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First Equilibrium Point. The linear equation system for the dynamics near the 
first equilibrium point Pi = (F,, Fj, F,) = (0, 0, 0) is then given by 



d_ 
dt 



^ y^-Y^\ T-IO 10 V 



72-^2 



R -\ 

-8/3 



^2-^2 



(9.151) 



The eigenvalues of this equation system follow from the extension of Eq. (9.21) to 
the three-dimensional case considered, 



0-r 


10 





R 


-1-r 











-8/3-r 



-(10 + r)(l + r)(8 / 3 + r) + 107?(8 / 3 + r) 



= -(8 / 3 + r)[(10 + r)(l + r) - 107?] = -(8 / 3 + r)[r^ + 1 1 r - 10(7? - 1)]. 

(9.152) 
This cubic equation for r has three roots that are determined by 8 / 3 + r = and 
the condition that the bracket term is equal to zero, 



11 121 ^^^„ ,, 11 
r^ = + J + 10(7? - 1) = 

' 2 V 4 2 



■ii-ji^.l0(7?-l)=-ii 
2 V 4 2 



1-J1 + — (7?-l) 

" 121 



(9.153) 



1 + J1 + — (7J-1) 
" 121 



The eigenvalues I'l and r^ are always negative. The sign of r, does depend on 7?: 
we have rj < if 7? < 1, and r^ > if 7? > 1. 

Second and Third Equilibrium Points. I. We can combine the stability analy- 
sis for the second and third equilibrium point by writing Eq. (9. 150) as 

.. ,. ( -10 10 V, 

d 



'y.-YA 



dt 



72-^2 

y.-Y,j 



y^-Y^ 



y^-Y, 
y.-Y.j 



-10 10 

1 -1 -^78(7?-l)/3 

^5^8(7? -1)/ 3 s^'&{R-\)l^ -8/3 

(9.154) 
The settings s=\ and i' = -1 correspond to the consideration of Pj ^nd P^, respec- 
tively. The eigenvalues of this system are determined by the condition 



(9.155) 



-10-r 10 





1 -1-r 


-.vV8(7?-l)/3 


.?V8(7?-l)/3 s^%{R-\)ll, 


-8/3-r 
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Table 9.2 Solutions y = r + a / 3 of the reduced cubic equation (9.159). The case a > implies 
D> 0. The last row provides the formula for the calculation of (p. 



a<0:D<0 



a<0:D>0 



a>0:D>0 



y\ -2£'cos 

J2 -2iiC0S 

J3 -2iicos 

(p coscp-- 



1. 
3 

(p + ln 

3 

(p + ^n 



2E' 



-lEcosh— 

3 

E cosh — + /v 3 sinh — 

3 3. 

E cosh / v3 sinh — 

3 3. 

n 
cosh^: 



2E' 



- 2E sinh — 
3 

<P 



sinh — + /v3 cosh— 
3 3, 

sinh / V 3 cosh — 

3 3 



sinh (p ■- 



J_ 

2E^ 



The expansion of the determinant provides the condition 

= -(10 + r)[(l + r)(8/3 + r) + 8(7?-l)/3] + 10[8/3 + r-8(7?-l)/3] 
= -(8/3 + r)[(10 + r)(l + r)-10]-8(7?-l)/3[l0 + 10 + r] 
= -r(8/3 + r)[r + ll]-8(7?-l)/3[20 + r] (9.156) 



2 8 + 33 ,,8 S(R-V)] 160(7?-!) 
-r\ r + r + ll- + -i ^' ^ ' 



3 41 2 8(7? + 10) 160(7?-1) 



3 3 3 

This eigenvalue equation is independent ois. Hence, this equation is the same for 
the equilibrium points Pj ^^id Py 

Solutions of Cubic Equations. Before analyzing the consequences of the last 
equation, let us briefly review the solutions of cubic equations. We consider the 
equation 

(9.157) 



= r^ +ar^ +br + c. 



Here, a, h, and c are any real coefficients. By introducing j = r + fl/3, Eq. (9.157) 
can be written as a reduced equation that does not contain a quadratic term. 



«H.-f .».-! 



■b\ y-—l + c 



= /-3/^ + 3y 



+ ay -2ay— + a\ — \ +by-h— + c 
3] 3 [3 3 



■y +y 



3\^]-6l^]\h 



3 / \3 

a \ .a 
+ 3 



7 « 

b— + c. 

3 j 3 



(9.158) 
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A more convenient way is to write this equation as 

Q^y"" +ay+l3, 
where a and /? are defined by 



(9.159) 



« =fo-3 



P = 2 



-b — + c. 
3 



(9.160) 



The three solutions of Eq. (9. 159) are given in Table 9.2 in dependence on 



D 




(9.161) 



Here, D is called the discriminant. The solutions of the original Eq. (9.157) can be 
obtained by means of the relation r^y - al3. Table 9.2 shows that there are two 
possibilities: there can be either three real solutions or one real solution and two 
conjugate complex solutions. The two solution regimes are separated at Z) = 0: 
for Z) < we have three real solutions, and for Z) > we have one real and two 
complex solutions. 

Pure Imaginary Eigenvalues. The two complex solutions may have positive 
or negative real parts. Regarding the evaluation of stability it is relevant to know 
for which R the real parts of complex roots are equal to zero (because we know 
then for which R the real parts of complex roots are positive and negative). This 
specific case of pure imaginary roots is given under the conditions that c = ab and 
& > 0. To prove the requirement of the first condition c ^ a bwe write Eq. (9. 157) 
for this case as 



= r^ +ar^ +br + ab = {r + a)(r^ +b). 



(9.162) 



This representation reveals the roots r, = -a and Tjj = ± / b^'^. The comparison of 
the solutions rj = -a and ^23 = + ; b^'^ for this case with the solutions presented in 
Table 9.2 shows that ii cosh(^/3) or Z'sinh(^/3), which represent the real parts of 
J2 and J3 depending on a negative or positive sign of a, respectively, must be equal 
to a / 3. These values imply zero real parts of r2 and r^ according to r^y- a/3. 
The second condition Z; > is a requirement to have a discriminant D> 0, which is 
needed for the existence of complex solutions. We have to calculate D to show the 
correctness of this claim. For c = a b, a and P are given by 



a =b-l) 



y^ = 2 



+ 2b- 



Hence, the discriminant D is given by 



D 



^y 



a 



\- 



+ b- 



(9.163) 



(9.164) 
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The evaluation of this expression provides 



Z) = 



+ 6 



+ 3- 



313 



+ 2h\ 



+ b' 



+ 9- 



313 



^ + 31" 
3 13 



(9.165) 



Therefore, Z) > under the condition that Z? > 0. 

Second and Third Equilibrium Points. II. The eigenvalue equation (9.156) 
can be analyzed on the basis of the solutions of cubic equations described in the 
preceding two paragraphs. Let us prepare this discussion by the calculation of two 
characteristic values oiR. A first characteristic value ^i is the value that separates 
three real solutions from one real solution and two conjugate complex solutions. 
^1 is determined by Z) = {aiyf + {filTf ^^. For our case, « and/? are given by 



a 



8(;? + 10) /41 



-R 



961 

27' 



P^l 



41 



9; 3 
8(7? + 10) 41 160(7? -1) 



1112 „ 10402 
-7? + - 



(9.166) 



27 



729 



The use of the latter relations in Z) = (aiyf + (Jil'If provides the expression 



D{K)- 



-R 



961 



556 „ 5201 
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27 729 
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72 



\iu9R.^]' 
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^R' 



R' 



72' 
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2 36,885 



32-108' 



32-108' 



^3 , 54,119 ^2 , 15,245 ^ 36,885 



96 16 16 

(9.167) 
The solution of cubic equations described above shows that the equation D(R) = 
has two negative real roots and one positive real root. The negative roots can be 
disregarded because we consider 7? > 0. The real root, which is 7?,, is given by 



R, 




cos 



1 



- arccos 



A^|33/2A 



2\a 



l3/2 



2;r 






■» 1.3456. 



(9.168) 



Here, a^ b - 3 (a / 3) and P= 2{a 1 3y - ab 1 3 + c, where a, b, and c are the 
coefficients of R^, R, and the last term in the cubic equation (9.156), respectively. 
The validity of this value may be seen by proving that Ri = 1.3456 implies Z) = 0. 
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Table 9.3 The linear stability properties of the Lorenz equations (9.145). 



Pi PjandPj 



<R< I asymptotically stable: do not exist 

3 real negative roots 

l<R<Ri unstable: asymptotically stable: 

2 real negative roots, 3 real negative roots 

1 real positive root 

Ri < R < R2 unstable: asymptotically stable: 

2 real negative roots, 1 real negative root, 

1 real positive root 2 complex roots with negative real parts 
R2<R unstable: unstable: 

2 real negative roots, 1 real negative root, 

1 real positive root 2 complex roots with positive real parts 



A second characteristic value R2 of R is the value that separates complex roots 
with negative and positive real parts. In terms of the notation of the general cubic 
equation (9.157), this value is determined by the condition c ^ a h. By adopting 
a, b, and c according to the eigenvalue equation (9.156), the condition for R2 reads 

41 8(^2+10) ^ 160(7?, -1) 
3 3 3 ' 

This equation is solved by 

470 
7?, =—^» 24.7368. (9.170) 

' 19 

Second and Third Equilibrium Points. III. The stability behavior near Pj and 

Pj can be determined now by taking reference to 7?i and T?,. We find the following 

features (see also the summary of linear stability features in Table 9.3): 

a) 1 < R < Rj] All the terms in Eq. (9.156) are positive for R> 1. Hence, all real 

solutions have to be negative. The discriminant D = («/ 3)^ + (/?/2)^ increases 

according to Eq. (9.167), dD / dR > 0, and we know that D = at 7?,. Thus, 

D <0 for 1 < R < Ri, which means that there are three real negative roots for 

1 <R<Ri. Hence, the solution near Pj and P3 is asymptotically stable. 

h) Ri < R < R2. For 7?, < 7? we have one real root, which has to be negative because 

all real roots must be negative, and two conjugate complex roots. The complex 

roots arise from imaginary contributions that appear in addition to the negative 

real parts of these roots. The real parts of these roots are negative when R < 7?,. 

Hence, the solution near Pj and P3 is asymptotically stable in this regime, too. 

c) R2 < R'- For R values in this regime we have one negative real root and two 

conjugate complex roots. The complex roots do have now positive real parts 

because of 7?, < R- Hence, the solution near P, and P3 is unstable in this regime. 
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9.5.3 Deterministic Chaos 

Equations Considered. The illustration of characteristic features of the Lorenz 
equations (9.145) will be focused on the case R2 < R. A discussion of solution 
properties for other cases can be found, e.g., in Sparrow (1982). In particular, we 
use R = 28, which means we consider the equations 

(9.171a) 
(9.171b) 

(9.171c) 

The reason for the consideration of these equations is that this case, for which we 
do not have asymptotically stable solutions, is the most interesting one. It will be 
shown below that the solutions exhibit a chaotic behavior in this case. This model 
may be considered as a highly simplified model for turbulence that is described by 
the Navier-Stokes equations. The condition R2< R does appear here in analogy to 
the condition for the onset of turbulence that the Reynolds number must be above 
a critical Reynolds number. 

Numerical Solution. Due to the nonlinear terms involved, the nonlinear equa- 
tion system (9.171) can only be solved numerically. For doing this we write these 
equations as a system of difference equations 

"^' :+lOAt(y2"-y:), (9.172a) 



dy,, 
dt 


-my^-y,). 


dyj 
dt 


= Ji(28-j3)-j2 


dy, 
dt 


8 

= J'lJ'2--J3- 



y\ =yi 



u+i n 

yi =yi 



■A?(j,"(28-j3")-j/), (9.172b) 



y^^ =j3"+A?UiV/--j3"J, (9.172c) 

where « = 0, 1, 2, ... Starting from the initial values {y^, jj", Ji) ^ C^io: yio^ V30), 
these equations describe the evolution of j,, yj, y^ in time t = n bd. The initial data 
(yio, j2o> J30) " (5, 5, 5) will be applied here (except for the study of the influence 
of varying initial data described below). Equations (9.172) introduces a parameter: 
the time interval Kt. This time interval is considered to be sufficiently small in 
order to produce solutions of Eqs. (9.172) that are independent of A?. In that case, 
the solutions of Eqs. (9.172) are seen as solutions of the differential equation sys- 
tem (9.171). The effect of different A? settings is illustrated in Fig. 9.9. This figure 
shows that the solutions of Eqs. (9.172) do not become independent of A? for A? 
variations over seven orders of magnitude (A? = 10"^ to A? = 10'°). It is not easy to 
consider the effect of smaller A^ values because such simulations are expensive. 
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Fig. 9.9. Solutions J' i(0 of the Lorenz equations (9.172) in dependence on the time 
interval At, where (y,o, J20' J30) " (5, 5, 5). The thick lines present the results for 
A?= 10 ^ 10 ^ 10 ', 10 ' in (a)-(d), respectively; the thin lines show the results 
for A? = 10-^ 10-^ 10-*, 10-'" in (a) -(d), respectively. 



The simulation, e.g., for A? = 10'" requires 11.1 hours on a Pentium(R) 4 CPU 
3.2 GHz personal computer with 1 GB memory. To improve the clarity of these 
plots, j'l is only shown from ? = 20 to ? = 40 in Figs. 9.9c-d. For t < 20, the thick 
and thin lines in these figures do hardly show any difference. The conclusion that 
the solutions of Eqs. (9.172) depend on the time interval At is surprising because 
this observation differs from the behavior of many other differential equations. 
It is relevant to note that this conclusion does not depend on the simple numerical 
scheme (9.172) used to solve the Lorenz equations (9.171): the result is the same 
for a variety of more advanced numerical schemes (see Yao 2007, 2010, Yao & 
Hughes 2008, and Liao 2009). Hence, it may be impossible to obtain a unique 
solution of the Lorenz equations (at least, no such solution that is independent of 
At has been reported so far). The Lorenz equations (9.145) have to be considered, 
therefore, as a guideline for the construction of numerical schemes (one possible 
numerical scheme is given by Eqs. (9.172)) that are defined in conjunction with a 
specific choice of At. This feature of the Lorenz equations also poses questions 
about the reliability of numerical solutions of the fluid dynamics equations (the 
Navier-Stokes equations): see Yao (2007, 2010). 
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Fig. 9.10. The evolution of variables determined by the Lorenz equations (9.172) 
in several phase planes, where A? = 10"^ and (ym, 720, J^so) " (5, 5, 5). (a), (c), and 
(e) show the evolution from ? = to ? = 20; (b), (d), and (f) show the evolution 
from ^= 10,000 to ^= 10,020. 



Phase Plane Evolution. The phase plane evolution of ji, y-^, and jj is shown in 
Fig. 9.10 for bd = 10 ^ which will be used in the following. These trajectories do 
never cross each other because the system never exactly repeats itself There is no 
convergence to any asymptotically stable state: the trajectories from t = 10,000 to 
t = 10,020 are very similar to the trajectories from ? = to ? = 20. The long-term 
behavior of the phase plane trajectories shown in these plots is called the Lorenz 
attractor, noted for the butterfly shape of the yx-y^ trajectory and the owl mask 
shape of the yx-y^ trajectory. 
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y^ 




Fig. 9.11. Solutions J i(0 of the Lorenz equations (9.172): the influence of varia- 
tions of initial conditions, where A^ = 10 '. The thick lines in these figures show 
the result for (y,o, y2Q, J30) " (5, 5, 5). The thin lines in (a), (b), and (c) result from 
the inidal data (5.01, 5, 5), (5.001, 5, 5), and (5.0001, 5, 5), respectively. 



The Butterfly Effect. The Lorenz equations are not only sensitive to variations 
of the time step used to integrate these equations. These equations are also very 
sensitive to minor variations of the initial conditions, as demonstrated in Fig. 9.11. 
In fact, every difference in initial data will result in different solutions. Figure 9.1 1 
demonstrates that the larger the difference of initial data, the sooner there will be a 
difference between solutions. Lorenz did accidentally discover this sensi-tivity of 
solutions to perturbations of initial data when he restarted the numerical 
integration of equations by rounding-off the data values used in the computations. 
The sensitive dependence of solutions on initial conditions is called the Butterfly 
Effect because Lorenz compared this dependence with the effect of a butterfly on 
the weather: he asked "Does the flap of a butterfly's wings in Brazil set off a 
tornado in Texas?". The Lorenz equations are clearly a highly simplified version 
of equations that can be used for weather forecasting. However, they may explain 
the reason of why long-term weather predictions are simply impossible: the non- 
linear interactions of variables involved in such equations imply a high sensitivity 
to perturbations, and perturbations have to be always taken into account (initial 
data are never known exactly). 

Probability Density Functions. The Lorenz equations are deterministic, but 
they produce output that looks like random data. It is, therefore, a reasonable idea 
to study the probability for finding certain solution values. This question will be 
addressed by considering the probability density function (PDF) of j,, y2, andjj 
values. The latter PDFs are denoted by/i(x,),/^(x2), and/^(x3), where Xi, Xj, and X3 
represent the sample space variables of ji, J2, and y^, respectively. The Lorenz 
equations were solved with A? = 10"^ up to ^ = 25, t = 100, and t = 1000. For t<\5 
the PDFs are still heavily affected by randomness: there are very sharp peaks that 
are difficult to resolve. The state ait = 25 corresponds to a state at which the PDF 
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Fig. 9.12. The probability density function (PDF)/i(xi),/;(x2), andy^(x3) related to 
ji, y2, and j3 values, respectively. The upper row shows these PDFs at t = 25, and 
the lower row shows the PDFs at ? = 100 (dashed line) and t = 1000 (solid line). 



is relatively smooth but still significantly affected by the initial data. The state at 
?= 100 represents the asymptotic state. Evidence for this conclusion is provided 
by the PDF at ? = 1000 which does not show an observable differences to the PDF 
at ?= 100. lO' solutions were generated by using varying initial conditions for j'l 
(with an equal distance) between 4.995 and 5.005. The PDFs were calculated as 
filtered PDFs according to the explanations in Chap. 4. A filter interval equal to 2 
was used to obtain smooth PDF curves. The results are shown in Fig. 9.12 for 
? = 25, t = 100, and t = 1000. The PDFs at ? = 25 are characterized by several 
modes (4, 5, and 3 modes regarding the/i,^^, andf^ curves). These PDF structures 
show that the initial data considered (which do only involve variations of ji 
values) may excite a spectrum of different motions. At ? = 100 (i.e., in the 
asymptotic stage) we observe a smoothening between these modes: different 
modes merge. The latter leads to the development of a central mode, but the outer 
modes are still present. The PDF curves seen here are clearly different from the 
features of velocity and temperature PDFs for the unstably stratified atmospheric 
boundary layer (only the/, PDF shows some similarities at t = 100), but the/j,/^, 
andfj curves are similar in the sense that they represent a superposition of several 
distinct motions. 
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9.6 Summary 

Let us summarize the observations made in this chapter regarding the extension 
of laws for one variable to the multivariate case of several interacting variables. 
This will be done by addressing the questions posed at the end of Sect. 9.1, i.e., 
the questions about the formulation of laws and the use of such equations. 

Formulation of Multivariate Laws. Newton's Laws of Mechanics apply to the 
case of several variables. Thus, there is no question about the laws for mechanical 
processes of macroscopic bodies that move with velocities much slower than the 
speed of light. With regard to the laws for population dynamics we have another 
case because there is no unique formulation of such processes (see the discussions 
in Chap. 7). In this case, we follow the spirit of formulating such laws by using 
empirical modifications of single-variable equations. The particular question with 
regard to both the laws for population ecology and mechanical processes is how it 
is possible to use such equations for several variables. This problem is much more 
complicated than the use of equations for only one variable. Let us summarize the 
findings obtained regarding this question. 

Numerical Solution of Multivariate Equations. The numerical solution of 
equations represents a general methodology for using evolution equations in order 
to study the features of processes. This approach does work if it is possible to find 
convergent solutions. A convergent solution represents a solution that is indepen- 
dent of variations of small time intervals used in the numerical scheme. For most 
equations it is possible to find such convergent solutions, but this is not always the 
case. An example for the latter case was given here by the Lorenz equations. So 
far, a convergent solution has not been reported for these equations. This finding 
does have implications for practical problems. The Lorenz equations represent a 
highly simplified version of the Navier-Stokes equations that are used to calculate 
fluid dynamics processes. With regard to most applications it is very expensive to 
prove the convergence of solutions to the Navier-Stokes equations (such simula- 
tions may need several years). Thus, solutions of the Navier-Stokes equations are 
calculated by adopting a relatively small time step. Then, there is the question of 
whether such solutions represent convergent solutions, which is not the case for 
the simple Lorenz model derived from the Navier-Stokes equations. 

Analytical Study of Multivariate Equations. It is hardly possible to integrate 
multivariate nonlinear coupled equations. Usually, analytical conclusions can only 
be derived by means of linear stability analysis. Such analyses are very helpful, as 
demonstrated for the examples given in this chapter. We obtain insight in this way 
that can hardly be obtained by numerical simulations. It would be scarcely possi- 
ble, e.g., to use numerical solutions for accurate calculations of the critical number 
7?2 that separates nonchaotic and chaotic solutions of the Lorenz equations. On the 
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other hand, linear stability theory does not represent an alternative to numerical 
solutions, because the overall features of nonlinear equations cannot be studied in 
this way. The analysis of nonlinear equations is only possible under very special 
conditions. An example was given here by the discussion of the application of 
Lyapunov's second method. Such methods are applicable if there is a way to find 
conserved variables, as, e.g., the total energy of a process. 



9.7 Exercises 



9.2.1 Consider the linear equation system 



d_ 
dt 



/, 



-1 2 
4 1 



/, 



The initial values are given by jiCO) = jm and j2(0) ^ J20) where jm and jjo 
are any parameters. 

a) Determine the solutions jv/O and>'2(0 in dependence on Jk, and j'2o- 

b) Which relation between the initial values >'io and>'2o is required such that 
J2 is a linear function of ji that disappears asymptotically? Find the cor- 
responding linear function 3^2 ^yiiyd- 

c) Which relation between the initial values jio and J20 is required such that 
_V2 is a linear function of ji that goes to infinity asymptotically? Find the 
corresponding linear function J2 ^yiiyd- 



9.2.2 Consider the linear equation system 

d_(yC\ ( 1 -^(1 + ^)' 

dtlyj 



'y.' 



The initial values are>'i(0) = 1 and>'2(0) ^ ~U and^ is any parameter. 

a) Find the solutions ji(0 and j2(0 to this initial value problem. 

b) For which range of ^ values do the solutions become zero as ? — > 00? 



9.2.3 Consider the nonlinear equation system 



diyi 
dAyi 



V 



a) Find the equilibrium points of this equation system. 

b) Which of the equilibrium points will be realized? Apply linear stability 
analysis to address this question. 
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9.2.4 Consider the nonlinear equation system 



d_(yi 
dt\y2 



yi yi 



yi 



a) Find the equilibrium points of this equation system. 

b) Which of the equilibrium points will be realized? Apply linear stability 
analysis to address this question. 

c) Apply the solutions yi{t) and yiit) to the nonlinear equation system to 
address again the question about the realization of equilibrium solutions. 
Hint: you may use the equation system to derive and solve equations for 
J, +J2 andji -y2, respectively. 

9.2.5 A specific form of Buffing's nonlinear spring model reads (Wiggins 2010) 



d(yA ( 1 



dt^yj) \}-y\ 






a) Find the equilibrium points of this equation system. 

b) Show that two equilibrium points represent centers. 

c) Use the linear equation systems near the centers to find the shape of 
trajectories in the yx-yi phase plane. Hint: you may calculate the ratio 
dyj dy2 = {dyj dt)l {dy2l dt) and solve the resulting separable equation. 

d) Explain the type of equation obtained for trajectories in c). 

9.3.1 Consider the following modification of the competition for food dynamics 
(9.73) discussed in Sect. 9.3.2 {d is a non-negative number), 

'^ = y,{l-y,-dy2), % = ;.,(l-^;.,). 

dt dt 

a) Find the equilibrium points of this equation system. 

b) Determine the stability behavior of solutions near the equilibrium points 
in dependence on the model parameter d. 

c) The coexistence equilibrium point depends on the value of d. For which 
range of d do we find a non-negative coexistence equilibrium point? 

d) Consider the range of values of d determined in c). Which equilibrium 
point will be realized asymptotically? 

9.3.2 Consider the following modified Lotka-Volterra equations for the prey ji 
(food fish) and predators jj (sharks): see Allen (2007), 

'^^yXA-f-By2), '^ = y^{-C - f ^ Dy,). 

dt dt 

Here, A, B, C, D, and/are non-negative constants. The parameter/models 
a prey reduction due to fishing. How does/affect a coexistence of species? 
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9.3.3 Infectious diseases such as measles, mumps, rubella, and chickenpox are 
modeled by involving three groups of individuals (Kermack& McKendrick 
1927, Anderson & May 1979a, 1979b, Anderson 1982, Fulford et al. 1997, 
Edelstein-Keshet 2005, Allen 2007). The total population A', which is con- 
sidered to be constant, is subdivided into susceptible (S), infective (I), and 
removed (R) classes: N^ S(t) + I{t) + R(t). Susceptible refers to individuals 
not infected but who are capable of contracting the disease and becoming 
infective. Infective refers to individuals who are infected and infectious. 
Removed refers to individuals who have had the disease and have definite- 
ly recovered, who are permanently immune, or are isolated until recovery. 
A very simple epidemic model (the SI model) assume 7? = and relates S 
and / by 

dt N dt N 

Here, y^is a positive constant of proportionality. 

a) Use the relation A' = 5 + / to derive a closed equation for /. Compare this 
equation with differential equations considered in Chap. 7. Which type 
of equation is the equation for II 

b) Solve the differential equation for /. 

c) Calculate the asymptotic values of S and / for large values of t. Explain 
the meaning of the result obtained. 

9.3.4 A modification of the equations described in exercise 9.3.3 is given by the 
following SIS model, 

dt N dt N 

Here, P and y are positive constants of proportionality. We have again the 
relation A'^ 5 + /. 

a) Explain the relevance of a nonzero y. 

b) Follow the approach in exercise 9.3.3 to derive a closed equation for /. 

c) Solve the differential equation for /. 

d) Calculate S and / for large values of t for the cases that /? > ^'and P< y, 
respectively. 

9.3.5 A modification of the equations described in exercise 9.3.3 is given by the 
following SIR model, 

^ = -^^7, '-L^^SI-yl, '-^-yl. 

dt N dt N dt 

Here, P and y are positive constants of proportionality. This model implies 
the relation N = S{t) + I{t) + R(t). 
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a) Explain the difference between the assumptions reflected in this model 
and the SIS model. 

b) Due to the relation A'^ = S{t) + /(?) + R{t) we can focus on the dynamics of 
S(t) and /(?). Use the model considered to derive an equation for the 
derivative dl I dS. 

c) Use this equation to calculate / = I{S). Rewrite this equation by using the 
abbreviations y = /?(/-/o) / (/N), x = 1 -S/Sg, and R = /iSJiyN). 
Here, /q and S^ refer to the initial values of/ and S, respectively. 

d) An epidemic occurs 'ify= y{x) increases from its initial value zero to a 
local maximum. Under which condition can this happen? Determine the 
critical point of x and the maximum of y. Hint: consider the fact that 
< X < 1 because 5 is a decreasing function by dSI dt = - PSIIN. 

e) No epidemic occurs if j is a decreasing function of x. Show under which 
condition this is the case. 

9.3.6 A modification of the equations described in exercise 9.3.3 is given by the 
following SIRS model, which is given by the equations 

dS P^j^ ^ dl p dR 

— = -^—SI-^vR, — = ^—SI-yI, — -yl-vR. 

dt N dt N dt 

Here, /?, y, and v are positive constants of proportionality. This model 
implies the relation Af= S{i) + I(t) + Rit). 

a) We can focus on the dynamics of S(t) and I(t) because R is determined 
via the relation A' = 5(0 + I(t) + R{t). Use the SIRS model considered to 
derive a closed equation system for S'(t) ^N- S(t) and I(t). 

b) Determine the equilibrium points implied by the equations for S* and /. 

c) Under which condition for the model parameters /?, /, and v involved do 
we find positive equilibrium values for S{t) and /(?)? 

d) Consider the case that the parameter condition derived in c) is satisfied. 
Which of the equilibrium points will be realized? 

9.4.1 Consider the total energy E ^ mg r(l - cos a) + m r^ {da I dtY 1 2. Use the 
fact that E is constant for the undamped pendulum to derive the differential 
equation for the undamped pendulum. Hint: differentiate E. 

9.4.2 The undamped nonlinear pendulum equation d^al dt? + sin « = combined 
with the initial conditions a(0) = a^ and da/ dt,(0) = can be used to find 
an exact expression for the pendulum period Tp, which is the time required 
for the pendulum bob to swing through one complete cycle and return to its 
original position. This expression for Tp reads (Boyce & DiPrima 2009) 



^. = 4Jt! 



"'2 d0 



g .^l-sin'(ao/2)sin'(9 
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The integral represents an elliptic integral of the first kind. 

a) Consider the case that the initial angle «o is very small. Calculate the 
integral by using the approximation (1 -x) "^ = 1 + x/2. Hint: use the 
integral |sin^X(ix = x/2-(l/4)sin2x. 

b) For which initial angles «o is the influence of «o on Tp smaller than 1%? 

9.4.3 Consider the undamped nonlinear pendulum equation (9.142), this means 
j2 = ±2'«(£, + cosj,-l)"l 

a) Write this formula in an explicit dependence on any initial conditions 
J'loandjjo. 

b) Explain under which conditions the positive and negative signs in the 
formula forjj have to be used, respectively. 

c) For which j'jo do we always find open curves in the yi-y2 phase plane? 

9.4.4 We consider a spring-mass system that in- 
volves two coupled masses m^ and Wj: see 
the corresponding illustration. The masses 
can move in one direction. Their positions 1 
are Xj and Xj. According to Newton's Sec- 



ond Law, this spring-mass system can be described by the equation system 
(Haberman 1977) 

rrii — J- - k{x2-Xi- L), Wj y~ ~ ~^ (■'■2 ~ -'^i ~ ^)- 

dt dt 

Here, k is the spring constant, and L is the unstreched length of the spring. 

a) The center of mass is defined by z = (wji Xi + /W2 -^2) / (wi + rrij). Find an 
equation for z and solve it. 

b) Consider the spring stretching >> = x, - Xj - Z. Derive an equation foxy. 

c) Compare the y equation with the undamped spring-mass system equa- 
tion d^y/df + ky/m = 0: see Eq. (7.42). Explain your observations. 

d) Solve the J equation. Hint: use the results derived in Sect. 7.3.3. 

9.4.5 Consider again the spring-mass system given in exercise 9.4.4. 

a) Solve the equations for Xi(?) and X2(t) for initial conditions chosen such 
that the initial values of the derivatives of z and j are zero (z'q ^ y'o = 0) 
and initial values of z and >> that are given by Zq = andju = 1. Hint: use 
the y and z solutions derived in exercise 9.4.4. 

b) Calculate the positions x,(^) and X2(t) for the case that m^ — > 00. Explain 
why the spring-mass system motion obtained in this way makes sense. 

c) Calculate the positions Xi(t) and X2(t) for the case that nii — > 0. Explain 
why the spring-mass system motion obtained in this way makes sense. 
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9.5.1 Consider the Lorenz equations (9.144). 

a) Calculate the equilibrium points by accounting for variable values of the 
model parameters Pr and b. 

b) Show that the characteristic number 7?,, which separates nonchaotic and 
chaotic solutions of the Lorenz equations, is given by 

' Pr-b-l 

Hint: follow the explanations in Sect. 9.5.2. 

9.5.2 Consider the expression for R2 given in exercise 9.5. 1 . 

a) Show the effect of increasing values of Z? on TJj. 

b) For which Pr values is R2 positive? 

c) Consider Pr values so that R2 is positive. For growing Pr values, Ri de- 
creases, it attains a minimum, and it increases. Find the minimum of 7?2. 

9.5.3 O. Rossler (1976) analyzed the following equation system (b represents a 
non-negative model parameter) 

%--J2-J3, % = j.+0.2j„ ^ = 0.2 + y,(y,-b). 

at at at 

a) Show that there exist two potential equilibrium points, which are given 
by P= (0.2 Y3, -Y„ Fj). Here, Y^ = (5/2) [b ± (Z;'- 4/25)"']. 

b) What are the conditions to have no equilibrium point, one equilibrium 
point, and two equilibrium points? 

c) Which behavior of solutions do you expect for the case that there is no 
equilibrium point? 

9.5.4 Consider the Rossler equations given in exercise 9.5.3. 

a) Determine the linear equation system near the equilibrium points. 

b) Show that the characteristic equation, which characterizes the behavior 
of the linear equation system obtained in a), is given by 

= r' + O.sib - 0.4 + ^b^ -4/25\^ + ilAb + 1 ± 2.6 V*'- 4/25 V 



+ V^'-4/25. 

The upper (lower) sign in this equation refers to the positive (negative) 
sign in 73 = (5/2) [^ ± {b"-'M 25)'% 
c) Show that the condition for the critical value of b that implies two pure 
imaginary eigenvalues is given by the equation 

= Vfe - 0.4 Ub(b^ - 0. 12) - 0.016 + ^^(fc' -0.12) + 0.016\ 
Hint: follow the explanations in Sect. 9.5.2. 
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d) What is the conclusion of the latter equation regarding the critical value 
of b that implies two pure imaginary eigenvalues? 

e) It was found for the Lorenz equations that the critical value of b that 
implies two pure imaginary eigenvalues separates complex eigenvalues 
with positive and negative real parts (nonchaotic and chaotic solutions 
of the Lorenz equations). Does the critical value of b determined in d) 
have the same property? 

9.5.5 Consider the Rossler equations in exercise 9.5.3 combined with b = 0.4. 

a) Use the results given in exercises 9.5.3 and 9.5.4 to find the equilibrium 
point and the roots r of the characteristic equation. Explain the stability 
behavior of solutions near the equilibrium point. 

b) Show the validity of the findings obtained in a) in terms ofyi-y2, yry^, 
and J2-J3 phase plane plots. Solve the Rossler equations up to ^ = 40 to 
obtain these figures. Use the initial values (vio, ^20^ J30) = (0.25, -1.05, 
0.95) and h.t = 10^ for the numerical solution corresponding to the 
numerical scheme (9.172) used for the solution of the Lorenz equations. 

9.5.6 Consider the Rossler equations in exercise 9.5.3 combined with b = 0.5. 

a) Use the results given in exercises 9.5.3 and 9.5.4 to find the first equilib- 
rium point and the related roots r of the characteristic equation. What do 
the results obtained mean regarding the stability behavior of solutions 
near the first equilibrium point? 

b) Show the vahdity of the findings obtained in a) hy yi-y 2, yy-y^, and V2-J3 
phase plane plots. Solve the Rossler equations up to ? = 40 to obtain 
these figures. Use the initial values (vio, yio, J30) " (0.35, -2.1, 1.9) and 
A^^IO"* for the numerical solution corresponding to the numerical 
scheme (9. 172) used for the solution of the Lorenz equations. 

c) Find the second equilibrium point and the related roots r of the charac- 
teristic equation. Explain the meaning of the results obtained regarding 
the stability behavior of solutions near the second equilibrium point. 

d) Show the validity of the findings obtained in c) by means oiy^y^, yx-y^, 
and j2->'3 phase plane plots. Solve the Rossler equations up to ? = 100 to 
obtain these figures. Use the initial values (yio, j'20) >'3o) " (0) -0.4, 0.4) 
and A? = 10 '' for the numerical solution. 
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The discussions of stochastic methods in previous chapters were related to the 
consideration of a single random variable. This approach is appropriate to explain 
the basic structure of evolution equations for stochastic processes and their PDFs. 
On the other hand, most applications cannot be handled on the basis of methods 
that describe the evolution of single variables. Real processes usually take place in 
the three-dimensional physical space, and they often involve several variables. 
Examples are given by flow phenomena (the three-dimensional atmospheric wind 
field that interacts with the temperature), chemical reactor processes (involving a 
variety of chemical species in three-dimensional reactors), and the competition of 
several population densities in areas with varying food resources. To prepare the 
application of stochastic methods to such cases we will extend now the methods 
developed in Chap. 8 to the case of several random variables. In fact, the methods 
to be described in this chapter are applicable to a wide range of realistic problems. 
More detailed descriptions of corresponding applications can be found elsewhere 
(Pope 2000, Roekaerts 2002, Heinz 2003, Fox 2003, Givi 2006). From a mathe- 
matical point of view, the discussion here reveals a relationship between partial 
differential equations and stochastic ordinary differential equations, which is very 
helpful for the solution of complicated partial differential equations. 

Section 10.1 explains the motivation for considering joint processes of several 
random variables. Joint PDFs that do not evolve will be considered in Sects. 10.2 
and 10.3: Sect. 10.2 explains the definition of joint PDFs and Sect. 10.3 presents 
the normal model for joint PDFs. Joint PDFs that evolve in time will be consid- 
ered in Sects. 10.4 and 10.5. The concepts for the description of the evolution of a 
single-variable PDF (and the corresponding stochastic process) will be extended 
to the several-variable case in Sect. 10.4. Section 10.5 explains the application of 
such equations to the modeling of molecular and fluid motion. Section 10.6 sum- 
marizes the basic observations made in this chapter. 
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10,1 Motivation 

Fluid Dynamics. As an example, let us consider the motion of fluids (e.g., 
atmospheric motions) in order to illustrate the need for methods for the calculation 
of the evolution of several random variables. The prediction of fluid flow requires 
the calculation of the mean velocity Ujix, i) of molecules, which represents the /"' 
component {i = 1, 3) of the fluid velocity at the position x = (xi, Xj, X3) at time t. 
It will be shown in Sect. 10.5 that the fluid velocity Uj{x, i) and fluid mass density 
p {x, t) have to satisfy a coupled system of partial differential equations, which 
represent the conservation of mass and momentum, 

^ + /,^^ = 0, (10.1a) 

Dt "^ dx„, 

^^ + 1^P5^ = 0. (10.1b) 

Dt p dx,„ 

Here, o;>„(jc, t) refers to the variance of molecular velocities, this means U;,,, = v-v^^ 
(see Sect. 10.5). We use the sum convention for repeated subscripts, this means 
we have for example 

^ = ^.^.^. (10.2) 

5x„, 5x, 8x2 Sxj 

The total derivative (or substantial or material derivative) of any property Q{x, t) 
(we may set, for example, Q^ pov Q^ U,) is defined by 

^^^.U,,.^. (10.3) 

Dt dt "'5x,„ 

The meaning of DQ I Dt can be seen by considering the property Q ?ii x = x{t). 
Here, x{t) is a point that follows the fluid velocity Ui, i.e., x{t) is determined by 

'^ = U,[x{t),t). (10.4) 

at 

The total derivative DQI Dt at x = x{t) reads 



DQ{x(t),t) _ dQ{x{t),t) ^ . x a6(x(o,0 
Dt dt '"^ ^ '' ' dx„, 

_ dQ{xit),t) ^ dQ{x(t),t) dx,„it) _ dQ{x{t),t) 
dt 5x„, dt dt 



(10.5) 



The last line makes use of Eq. (10.4). Hence, DQI Dt represents the total change 
of the property Q in time at a point xit) moving with the fluid velocity C/,. 
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Closure Problem. Equations (10.1) are unclosed because the variance a;,,, of 
molecular velocities is unknown. This is not a minor problem, but C!;„, determines 
the velocity change DUjI Dt according to Eq. (10.1b). The variance o;,,, has to sat- 
isfy a conservation equation, too (see Sect. 10.5), 



Dt p dx„, 5x,„ - dx,„ "" t['^ 3 " 



Here, v,v v,„ is the triple correlation of molecular velocities, T is a characteristic 
relaxation time scale, and Sjj refers to the Kronecker delta (which is zero for / ^J 
and one for / =j). This equation is again unclosed because the triple correlation is 
unknown. It would be possible to continue in this way by considering an equation 
for the triple correlation. However, this equation does again contain an unknown 
correlation of higher order, and this applies to all such equations. The solution of 
this closure problem requires a model that explains the evolution of all moments 
of molecular velocities, which define the joint PDF of the three molecular velocity 
components. Therefore, we need a model for the evolution of this joint PDF. Such 
a model will be presented in Sect. 10.5. 

Questions Considered. Hence, we have to extend the methods for the analysis 
and modeling of single random variables to the description of properties of several 
random variables. In particular, we need answers to the following questions: 

• How can we extend concepts for the data analysis of single random variables to 
concepts for the data analysis of joint random variables? 

• How can we extend usual PDF models for single random variables (for example, 
the normal PDF model) to the case of several variables? 

• How can we extend PDF equations for the description of the evolution of single- 
variable PDFs to equations for the evolution of joint PDF of several variables? 

The first two questions will be considered in Sects. 10.2 and 10.3 for the case of 
two random variables by focusing on the data analysis. The last question will be 
addressed in Sects. 10.4 and 10.5 with focus on the modeling of several-variable 
processes. 



10.2 Data Analysis Concepts for Joint Random Variables 

How can we extend concepts for the data analysis of single random variables to 
concepts for the data analysis of joint random variables? First of all, this requires 
the definition of a joint PDF, this means the explanation of how a joint PDF can 
be obtained from measurements. It will be also helpful to extend the definitions of 
a single-variable PDF and its moments introduced in Chap. 4 by the consideration 
of correlations, which corresponds to the introduction of conditional means. These 
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questions will be addressed in this section by considering PDFs of two random 
variables X and Y, which may have values between negative and positive infinity. 
The concepts to be developed can be straightforwardly extended to the case of 
many variables. Such multidimensional joint PDFs will be considered in Sect. 
10.4.1 in the context of the discussion of evolution equations for joint PDFs. 



10.2.1 Joint Probability Density Functions 



Joint PDF. In extension of the definition /(x) = <d{x - X)> of the PDF of a 
single random variable X, we define the joint PDF of two variables X and Y by 



f{x,y) = {5{x-X)5{y-Y)). 
The joint PDFy(x, y) has the properties 

\f{x,y)dy = \{S(x - X)S(y - Y))dy ={5{x - X)) = f{x). 
\f{x,y)dx = \{5{x - X)5{y - Y))dx ={5{y - Y)) = f{y). 



(10.7) 

(10.8a) 
(10.8b) 



The first rewriting of the left-hand sides makes use of the definition (10.7) of the 
joint VDV j{x, y). The second rewriting applies the normalization property of delta 
fiinctions. The PDFs/(x) anAfiy) of single variables are called marginal PDFs. As 
shown in exercise 10.2. 1, other typical properties of the joint VDY J{x, y) are 



/(-oo, j) = /(oo, j) = /(x ,-oo) = /(x,oo) = 0, 
\\f{x,y)dxdy = \, 
\\g{x,y)fix,y)dxdy = {g{X,Y)), 



(10.9a) 
(10.9b) 
(10.9c) 

(10.9d) 



where g(x, y) is any function of x and y. The knowledge of the joint PDFy(x, y) 
enables the calculation of the probability for joint events a <X<h and c <Y<d, 



P(a<X<b,c<Y<d) = \ 



\fix,y)dx 



dy. 



(10.10) 



The validity of this relation can be seen by using the definition (10.7) ofjix, y), 



\fix,y)dx 



dy = {\ 



'i ddjx-X) d9{y-Y) 



dy 



dx dy 

= {{e{b -X)- e{a - X)) {e{d -y)- e{c - y))). 



(10.11) 
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Here, the delta functions were replaced by derivatives of theta functions according 
to 3(x-X) = d9{x-X)l dx. Relation (10.10) can be specified for the case 



y+dy 

P{x<X <x + dx,y<Y <y + dy)^ \ 

y 



\f(x,y)dx 



dy, (10.12) 



where dx and dy are infinitesimal intervals. In the first order of approximation we 
can replace f{x, y) in the integral by^^, y). Then, Eq. (10. 12) provides 

P(x <X<x + dx,y <Y<y + dy) = f(x,y)dxdy. (10.13) 

Hence, J{x, y) determines the probability to find X and Y in infinitesimal intervals 
at X and J. 

Independence. The joint PDF J{x, v) becomes simpler for the specific case of 
independent random variables, this means for the case that there is no effect of one 
variable on the other variable. For this case, the joint PDF can be written 

fix, y) = {Six - X)) {Siy - Y)) = fix) fiy). (10. 14) 

The consideration of independent variables simplifies analyses significantly. The 
concepts of independent and uncorrelated random variables (variables with a zero 
correlation coefficient rxY, see Sect. 2.3.1) are similar but different. Independent 
variables are always uncorrelated: the correlation coefficient r^= 0. However, the 
converse is not true in general: uncorrelated variables do not have to be independ- 
ent. An example for the latter case is the following: Let X be uniformly distributed 
on [-1, 1] and Y = X^. The calculation of the correlation coefficient then shows 
that both variables are uncorrelated. However, X determines Y, and 7 restricts X to 
at most two values. Hence, X and Fare not independent variables. 



10.2.2 Conditional Probability Density Functions 

Conditional PDF. The joint PDF f^x, y) determines the probability to find X 
and Y in infinitesimal intervals at x and y. However, there is relatively often a 
slightly different problem given by the question of what is the probability to find 
values of one variable (e.g., y) for a fixed value of the other variable (e.g., x): see 
the discussion in Sect. 10.2.3. Information regarding this question is given by the 
joint PDF fix, y), but/(x, y) does not represent a PDF for y (the integral over y 
does not result in one: see Eq. (10.8a)). Therefore, the joint PDF is rescaled so that 
the rescaled PDF integrates to one. This rescaled PDF is given by 

/(J|x) = ^^- (10.15) 

fix) 
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The PDF Jiy \ x) is called the PDF of y conditioned on x (or simply conditional 
PDF): it describes the probability to find y values under the condition that X= x. 
The integral of the conditional VDV f(y\x) overjv is equal to one, 

\f{y\x)dy^\, (10.16) 

which follows from/(y | x) ^ J{x, y) I J{x) and the property (10.8a) oiJ{x, y). The 
conditional PDF of independent variables, for which we have J{x, y) = f{x)f{y), is 
equal to the corresponding unconditional PDF,/(>' |x) =J{y)- 

Conditional Mean. The conditional PDF can be used to define a conditional 
mean. With regard to any function g{x, y), this relation reads 

\gi^,y)fiy I x)dy = {g(x,Y) \ x). (io.i7) 

By using the definitiony(y | x) =J{x, y) lf(x), this relation also can be written 

\g{x,y)f{x,y)dy = {g(X,Y) \ x)f(x). (10.18) 

The consistency of this relation can be seen by integrating it over x, 

j{g{X,Y) I x)f{x)dx = jjg(x,y)f{x,y)dydx = {giX,Y)). (10.19) 

The last expression follows from the property (10.9d) of joint PDFs. Hence, the 
integral over the conditional mean multiplied with the probability to find x equals 
the unconditional mean. 

Conditional Mean Calculation. Equation (10.17) can be used to calculate a 
conditional mean, but this requires the joint PDFy(x, y) and the integration over j. 
This can be avoided by performing the integration over j in Eq. (10.17), 

{giX,Y)\x) = -^jgix,y){S(x-X)Siy-Y))dy 
fix) 

= ^{giX,Y)\S{x-X)S{y-Y)dy) = -^{giX,Y) S{x - X)). 
fix) ^ ' fix) 

(10.20) 

The first rewriting of the conditional mean is obtained by replacing the joint PDF 

fx,y) in the conditional PDF/(y | x) ^fx, y) /fx) by its definition (10.7). In the 

second line, the mean value is used for all the integral, and the sifting property of 

delta functions is used, such that g(X, Y) can be written in front of the integral. The 

last expression results from the normalization condition for delta functions. The 

relation between a conditional mean and a conditional PDF can be seen by setting 

g(X, Y)^ d(y- Y). For this case, Eq. (10.20) will become 

{S(y-Y)\x) = -^{S(y-Y)Six-X)) = ^^^ = fiy\x). (10.21) 

fix) fix) 
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These rewritings follow from the definitions of a joint VDF J{x, y) and conditional 
PDF J{y I x). Thus, the conditional PDF represents a conditional mean. 



10.2.3 Application to Optimal Modeling 

Optimal Models. Typical problems involving two random variables were 
considered in Chap. 2. We considered a set of (A^, 7,) data, where / = 1, A'. The 
problem was to find a model yU.^) that agrees as good as possible with the given 
data. The particular problem was to find a model yiJ^x) that minimizes the least- 
squares error 

E'=^t{Y,-yM(^.)f- (10.22) 

The objective here is not to make any modifications of the approach presented in 
Chap. 2, but to present the findings obtained in Chap. 2 in terms of properties of 
random variables. 

Error Definition. In terms of the notation applied here, the least-squares error 
can be written as a mean value, 

J^'={[Y-yM(^)Y\ (10.23) 



According to Eq. (10.19), the least-squares error E^ also can be written in terms of 
a conditional mean, 

E'=\{[Y-y,,iX)y\x)f(x)dx = \{[Y-y,,ix)y-\x)f{x)dx. (10.24) 



The last expression accounts for the condition X^ x. The advantage of using the 
conditional mean is that the error E^ is now related to the function yuix), which 
has to be calculated. 

Minimal Error. Which model function y/Xx) could minimize the least-squares 
error? The last expression in Eq. (10.24) represents the mean value of the non- 
negative numbers <[Y - yii/x)Y \x>. Therefore, the minimum value of ii" is given 
if <[F- j^x)]^ I jc> becomes minimal. This conditional mean can be written 

[Y - y^ (x)]^ I x) = {[y -{Y\x) + {Y\x)- y,, (x)] ' \ x) 

, (10.25) 

[y-{Y\ x)] ' I x) + ([(y I x) - y,, (x)] ' I x) + h{x). 

The first rewriting involves <Y\ x> - <Y\ x>. The second rewriting results from 
distributing the quadratic term, where the function h{x) is given by 

h{x) = 2{[y-(y I x)][(y | x)-y^{x)]\x). (10.26) 
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A closer look at h{x) shows that h{x) = 0, 

h{x) = 2[{Y\x)-y,{x)\{[Y-{Y\x)]\x^ ^^^ ^^^ 

= 2[{Y\x)-y^ix)][{Y\x)-{Y\x)\=0. 

The expression <Y \ x> - yiJ^x) is unaffected by the condition, which results in the 
first line. The next line follows from distributing the conditional mean. Therefore, 
Eq. (10.25) can be written 

[Y-y^ix)Y\x) = {[Y-{Y\x)y\x) + [{Y\x)-y„{x)Y. (10.28) 



The model function yiJ,x) does only affect the last term, which is non-negative. 
Thus, the conditional mean <[y - Ja/^)]^ | x> becomes minimal if the last term 
disappears, this means if 

y^ix) = {Y\x). (10.29) 

This expression is relevant: (/) it explains how an optimal model function j^v/x) 
can be calculated on the basis of measured data (without making use of any model 
assumptions), (//) it provides a basis for the optimization of model function types 
considered (see the explanations in the next paragraph), and (///) it enables the 
calculation of y^x) on the basis of a model for the joint PDF/(x, y) of X and Y 
(see Sect. 10.3). 

Optimal Linear Model. The usual way to address optimization problems is the 
attempt to transform the data such that a linear model can be used, 

yn^{x)^ax + b. (10.30) 

The model parameters a and b can be calculated in terms of Eq. (10.29). By using 
yj^x) = <y|x> and the definition (10.20) of conditional means, Eq. (10.30) multi- 
plied byy(x) can be written 

{YSix-X)) = {ax + b)f{x). (10.31) 

We take the integral over x to obtain a condition for b, 

{Y) = a{X) + b. (10.32) 

By replacing the parameter b in Eq. (10.31) by this condition we obtain 

{Y5{x - X)) = {Y)f{x) + a{x - {X))f{x). (10.33) 

The term <Y>j{x) can be combined with the left-hand side, 

l^Y5ix-X)) = a[x-{X))f{x). (10.34) 
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The multiplication of this expression by x - <X> and integration over x then 
provides a condition for the model parameter a, 

alx") = l\[x- iX)) Y 5{x - X) dx) = {\[x- (X)) Y 5{x - X) dx) 

(10.35) 
= (p( - {x))y\5{x - X)dx^ = {^XYy 



Here, we used the sifting property and normalization condition of delta functions. 
By combing jv^(x) = <y|x> ^ ax + b with Eqs. (10.32) and (10.35) we get 



- - - ^. (10.36) 



y^ (x) = {Y\x) = (Y) + A— i(x - {X)) = (Y) + r„, {y' ) 
r / 



X' 



This result recovers Eq. (2.47). The last expression applies the correlation coeffi- 
cient, which was already defined in Chap. 2, 



{H) 



,1/2 ,~ il/2 • 

X'\ Y^^ 



(10.37) 



The use of the latter expression for^-^x) in Eq. (10.23) results in the following 
minimal least-squares error 



E'={[Y-y^ (X)] ') = ([?- x{xy) i{x'- 



IxY) (xY) 

= {Y')-2\^{xy) + ^^{X') (10.38) 



= (^")-tel-(?1(l-'--^)- 



This expression recovers Eq. (2.53) forii^, see the related discussion in Chap. 2. 



10.3 The Joint Normal Pobability Density Function Model 

Let us address now the question of how joint PDFs can be modeled. We will 
consider here the extension of the normal PDF model for single variables, which 
represents the most relevant PDF model for unbounded variables, to the case of 
two correlated random variables. The extension to the many-variable case will be 
described in the context of Fokker-Planck equations (see Sect. 10.4). 
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10.3.1 The Joint Normal Probability Density Function Model 

Joint Normal PDF. The joint normal PDF f{x, y) of two random variables X 
and Zcan be defined by 

fix, y) = — I , , , . exp 



2^^i\-r^')(x'){P 



2^^(1-J)(x^>(f^)"'1 2(l-r^^) 



X 


+ y' 


-Ir^iy 






2(1- 


-rxY^) i 




(y 


-'"^•- 


xf+i^-rxY 


')i'' 



(10.39) 



The second line represents a convenient rewriting of the first line, which will be 
used below. To represent these expressions efficiently we apphed here the non- 
dimensional variables 

- x-(x) . y-(Y) 

^ = 7^' 3^ = 73-VT7r (10-40) 

x^) (y^ 



as abbreviations. The correlation coefficient r^fy is given by Eq. (10.37). By defin- 
ing normalized random variables 

^ = 7^77. ^ = 77^ (10.41) 

x^) (y' 



in analogy to Eqs. (10.40), we find the correlation coefficient r;^, to be given by 

(xy). (10.42) 



Due to \rx}\ < 1 we have 1 - i'xy^ > 0, i.e., the variance in Eqs. (10.39) is non- 
negative. The model (10.39) does satisfy the consistency conditions (10.8), see 
exercise 10.3.1. 

Moments. An efficient way to present the moments of the joint PDFy(x, y) is to 
do this in terms of the normalized random variables (10.41). The moments can be 
calculated by multiplying the PDF J{x, y) with the corresponding variables and 
integration. Similar to the properties of a single-variable normal PDF it is found 
that the third-order and fifth-order (and all other odd-numbered) central moments 
are equal to zero, 

x') = {x'y) = (xy'] = (y') = o. 



(10.43a) 

x') = {x'y) = (x'P) = (x'P) = (xy'' '^" 
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The even-numbered normalized central moments are functions of the correlation 
coefficient r^y. For example, the fourth-order and sixth-order central moments are 
given by 



X'Y) = (XY') = 3ryy, (l'y) = (ly') = 15/- 



(10.43b) 

X^Y^) = \ + 2r^\ (x'Y^) = {x'Y') = 3 + nr^\ 

X'Y') = 9r^+6r^\ 

For independent variables for which we have i'xy ^ 0, these relations recover the 
consequences for single normally distributed variables. Equations (10.43) can be 
used to decide whether any joint PDF is normal or not (see, e.g., the discussion of 
this question regarding the Brownian motion model in Sect. 6.4.2). For the case 
that all the conditions (10.43) implied by a joint normal PDF are satisfied, we can 
conclude that the joint PDF considered represents a normal PDF. Why is this 
conclusion valid? It is possible that another joint PDF implies moments that agree 
with some of the relations considered here (this PDF may also imply zero third- 
order and fifth-order moments), but it is impossible that another joint PDF implies 
moments that agree with all the 22 conditions (10.43). 

Conditional PDF. Equation (10.39) can be used for writing the joint PDF as 

fi^,y)= I ^ , exp p£""^^]% /(x), (10.44) 

/2;r(l-r^^)(y^) i 2(1 -r^^) 

where /(x) is given by 

/(x) = ^^=expj-4l- (10.45) 

2n:(^X^ 

Comparison of Eq. (10.45) with the definition of the conditional PDF/(y |x), i.e., 

f{x,y) = f(y\x)fix), (10.46) 

shows that the conditional PDFJ{y \ x) is given by 

1 I (y-rxY^y 



/2;r(l-r,/)y^ 1 2(1 -r^-)^^ 



The conditional PDFy(y |x) integrates to unity, |/(v|x) dy^ 1, see exercise 10.3.2. 
Considered as a function of y ,fiy\x) represents a normal PDF with mean r^ x 
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and variance 1- Vxy , which is divided by < 7^ > . Hence, we have the relation 

ji)/(j|x)«'j) = -^^. (10.48) 

Conditional Mean and PDF. In terms oijly \ x) we can obtain all conditional 
moments. First of all, we are interested in the conditional mean, which is given by 

{Y\x) = \yf{y\x)dy. (10.49) 

The conditional mean can be calculated by writing Eq. (10.48) as 

S^nyU^^-jS^- (10-50) 

(7') (7') (7' 



By using the definition (10.49) we find then for the conditional mean 

(y I x) = (7)+ r^,(y^)'"x = (7) +r^(y^)'" ^^, (10.51) 



where x is used according to its definition (10.40). This expression for the condi- 
tional mean enables us to write the conditional PDF/()'|x) given by Eq. (10.47) in 
a very convenient way. To prepare this representation we write 

, J -(7) (7|x)-(y) >'-(y|x) 

FM (F'\ (F' 



where the definition of y and expression (10.51) for the conditional mean are 
applied. The use of this relation in Eq. (10.47) leads to the conclusion that 

f{y\A = ^^ ^ , . exH- " ^': ;:,, K (10.53) 




2;r(l -r„-)(F 

Therefore, the conditional PDF represents a normal PDF with mean <F | x> and 
variance (1 - r^y*) <F^>. Thus, the deviations F- <F| x> from the conditional 
mean are normally distributed with zero mean and variance (1 - r^~) <F^>, this 
means the deviations F- <F|x> are independent of x. 

Statistical Formulation of Optimal Models. In Sect. 10.2.3 we analyzed the 
consequences of considering a linear conditional mean, which leads to the global 
variance (10.38). Evidence for the suitability of considering such a mean and 
variance was not provided, which leads to the question of whether there is any 
conditional PDF that has such a mean and variance, and whether it is reasonable to 
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consider such a PDF. Answers to these questions are obtained by the findings 
obtained in the previous paragraph. We see that the assumption of a joint normal 
PDF, which is certainly reasonable, implies a linear conditional mean. The global 
variance (10.38) is also supported by this PDF, see exercise 10.3.3. 



10.3.2 Data Analysis 

There is often the question of whether a joint normal PDF can be applied to 
model the joint PDF of given X and Y data. This question cannot be answered by 
considering only the marginal PDFs of X and Y. For example, it is incorrect to 
conclude that two variables have a joint normal PDF even if the marginal PDFs of 
bothXand Fare normal PDFs: the mixed moments may differ from Eq. (10.43b). 

Joint Normal PDF Features. To address this question, it is helpful to know 
the characteristic features of a normal joint PDF. The best way to illustrate the 
joint PDF features is to consider isolinesy(x, y) =/in the x-y plane, where/is a 
constant. In this case, we can write Eq. (10.39) as 



= -(1 - r^,')ln(4;r V' (1 - '-^')(^')(?'))- -(1 - '-^')ln[(l - r;^') C], 

where the constant C is defined by C = 4 n:^f^ <X^><Y^>- The meaning of this 
relation can be better seen by introducing the variables 

x'=^, y=^. (10.55) 

The (x', >'')-coordinate system is obtained by rotating the (x, y) -coordinate system 
by an angle of 45°. The variables x and y are related to x' and y' by 

x = -^, y = -^. (10.56) 

In terms of the latter expressions we can write the left-hand side of Eq. (10.54) as 

i^+r-2r^.iy = \[{x'-y'f + {x'+ff -2r^.{x'+y'){x'-y')] 

^ (10.57) 

= ^[2x''+2y'-2r„{x''-y')]={\-r^.)x''+{\ + r^)y\ 

such that Eq. (10.54) reads 

(l-r_^)x'^ + (l + r_^)y^ = -(l-r_^')ln[(l-r_^')C]. (10.58) 
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Fig. 10.1. Isolines of the joint normal PDF/(x, j). The 
(x', j')-coordinate system is obtained by rotating the 
(x,y) -coordinate system by a 45° angle. The isoline 
J{x, y) =/ is an ellipse in the (x', j')-system. Here, a is 
the semimajor axis, and b is the semiminor axis. 



The latter equation can be written 

:1. 

Here, the parameters a and b are given by 



a b 



-•'- 


-''xy')ln[(l- 


-rxY 


^)C] 


V i-<^„ 


RT 


-r^,^)ln[(l- 


-rxY 


^)C] 



V-ln[(l- 



)C] 



l+rxY 



1 + r, 



V-ln[(l- 



'-)cV 



(10.59) 



(10.60a) 



(10.60b) 



The relevance of Eq. (10.59) is that this relation represents an ellipse equation: see 
the illustration in Fig. 10.1. This ellipse equation involves two specific cases. For 
Tjfy = we find a = b = [-In C]"^, which means that the ellipse becomes a circle. 
The second case is given for r^Y = ±1: for Vxy ^- +1 we have a line along the x' 
axis, and for rjn'— ^ -1 we have a line along the j' axis (see exercise 10.3.4). 

Filtered Joint PDF Calculation. How can we numerically calculate the joint 
PDF /(x, y) to test the suitability of model assumptions? In extension of the cal- 
culation of marginal PDFs we calculate the filtered joint VDV fj^x, y) by 



fAi.x,y) = 



1 AJV,, 
Ax Ay A^ 



(10.61) 



Here, A/V^,, is the number of {X, Y) realizations that are found in x and y intervals 
centered at x and j. This means, A/V„, refers to the number of (X, Y) realizations for 
which X and 7 satisfy the conditions 



Ax „ Ax 

X <X <x + — 

2 2 



and 



2 2 



(10.62) 



It is relevant to note that X and Y are not any random values, but they represent a 
joint event (they are measured at the same time). 
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Fig. 10.2. Scatter plots of jointly normal and non-normal random variables. Here, 
II , w , and T are standardized velocity components u and w and temperature T 
under neutral conditions, see the explanations given in Sect. 4.5. The correlation 
coefficient r,„,, = -0.26 in (a) and (c), whereas r„j- = 0.39 in (b) and (d). The scatter 
plots in (a) and (b) show jointly normally distributed random variables, and the 
scatter plots in (c) and (d) are obtained from measurements described in Sect. 4.5. 
The solid lines represent isolines of jointly normally distributed variables. The 
outer and inner isolines correspond to/* 0.02 and/* 0.1, respectively. 



Scatter Plots of Normal Variables. A good way to illustrate joint PDFs is to 
consider scatter plots of joint PDF isolines. Such scatter plots can be obtained by 
presenting all (X, Y) positions for which the joint PDF fj^x, y) has a certain value 
(or is found inside a certain interval). An example for this way of looking at the 
joint PDF is given in Figs. 10.2a-b. These figures show scatter plots of jointly 
normally distributed random numbers. These examples are set up according to 
Figs. 10.2c-d. Therefore, the variables are called ii , w , and T {ii and w refer 
to velocity components, and T refers to the temperature). The mean of these vari- 
ables is zero and the variance is one, this means we consider standardized random 
variables. Figures 10.2a-b differ by their correlation coefficients, which have 
values that agree with the values in Figs. 10.2c-d. The joint PDF was calculated 
by using Ax = Ay = 0.2. As used for the Figs. 10.2c-d, a total number of 50,400 



406 10 Stochastic Multivariate Evolution 



random numbers was considered. The outer isolines correspond exactly to the 
constant value/= 40 / {N Ax Ay) « 0.02, and the inner isolines correspond exactly 
to the constant value /= 200 / {N Ax Ay) « 0.1. Figures 10.2a-b show that the 
scatter plots obtained in this way agree very well with the isolines, which were 
calculated according to Eq. (10.59). 

Scatter Plots of Non-Normal Variables. Such scatter plots of joint PDFs can 
be used to test the suitability of modeling measured data by a joint normal PDF. 
An illustration of this approach is given in Figs. 10.2c-d. These figures show joint 
PDFs of measured velocities and temperatures that were used in Sect. 4.5 to study 
marginal PDFs derived from measurements. We see here the joint ii-w PDF and 
the joint ii-T PDF for a neutral stratification. These joint PDFs have been calcu- 
lated in the same way as the joint PDFs in Figs. 10.2a-b. The marginal PDFs of m 
and w shown in Fig. 4.17 reveal that both PDFs can be described very well by a 
normal PDF. Hence, the scatter plot in Fig. 10.2c agrees very well with the 
corresponding plot in Fig. 10.2a, which means that the joint ii-w PDF can be 
described very well by a joint normal PDF. As may be seen in Fig. 4.17, the 
marginal temperature PDF can be described only approximately by a normal PDF. 
Hence, the scatter plot in Fig. 10. 2d also shows deviations to the corresponding 
joint normal PDF features given in Fig. 10.2b. Nevertheless, regarding the usual 
lack of alternatives it is still reasonable to describe the joint u-T PDF by a joint 
normal PDF. 



10.3.3 Application to Random Walk Modeling 

Let us consider the modeling of random walk (see Sect. 6.3) to illustrate the 
application of concepts introduced above. We consider a random variable (e.g., 
the position of any object) that is initially normally distributed. In each time step, 
the variable changes by the addition of a normally distributed contribution, which 
is independent of previous values of the random variable (it is worth emphasizing 
that the result to be obtained below can be extended to the case of jointly normally 
distributed variables that are correlated: see exercise 10.3.6). Hence, the random 
variable considered represents at every time a sum of independent nonnally distri- 
buted random numbers. The question related to this problem is to find the PDF of 
the variable considered at any time, this means the PDF of a sum of independent 
and normally distributed random numbers. This question, which requires the use 
of joint PDF concepts due to the need to consider simultaneously various random 
variables involved in the sum considered, will be addressed in the following. 

Sum of Two Variables. First, let us consider the sum of two random variables 
with any statistical properties. In particular, we consider one variable X, with a 
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marginal PDF/J(xi), and another variable Xj with a marginal PDFy^(x2). The joint 
PDF of both variables is given by/i2(xi, X2). Our objective is to calculate the PDF 
y(z) of the sum Z = Xy +X2. For doing this it is helpfiil to consider the distribution 
function F(z), which enables the calculation of the PDF by means oij(z) = dF I dz. 
The distribution function F{z) of the sum of two variables can be related to the 
joint PDF/i2(xi, Xj), which is considered to be known, by the relation 

m --.T| 

F{z)= I j /,2(xj,X2)(iiC2 '^■*^i- (10.63) 

Evidence for the validity of this relation can be obtained in the following way, 

-» \ dxi j (10.64) 

/-|-^^(x.-X.)^^^ 

\ -GC UX^ 

= {9iz-X,-X,)-9{-^-X,)) = {9{z-X,-X,)). 

The first line applies the definition of/i2(x,, X2). The integration with regard to X2 is 
performed in the second line. The third line accounts for 9 (-00 -X^ = and the 
fact that the integral is only nonzero if x, < z -X2. The brackets have to apply to 
all the integral now because the upper bound is a random number. The integration 
with regard to Xi is performed in the fourth line, where 9 (-00 -X,) = is used. The 
last expression represents P{Xi + X2 < z), which is the definition of F{z). The 
corresponding PDF can be obtained by differentiating F{z), 

f(z) = ^^= J /,2(x„z-x,)Jx,. (10.65) 

dz _„ 

For the case that Xi and X2 are independent, the last formula reads 

/(z)= J/,(x,)/2(z-x,)fl'x,. (10.66) 

Sum of Two Independent Normal Variables. Next, let us apply the definition 
(10.66) of y(z) for the case that X^ and X2 are independent normally distributed 
random variables. The use of the normal PDF expression (4.72) results in 

/(z) = -^J exp|-%4l-ii^|H^Lx, (10.67) 

2;rc!"iC!"2 -X 2(71 ^^2 
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Here, //j and a^ are the mean and standard deviation of Xi, and //2 and CTj are the 
mean and standard deviation of X,. By introducing y = Xi - /U^ and replacing the 
integration over Xi by an integration over y,J{z) is given by 



f(^) 



1 



2;rcrj c^ _, 



j expi 



-^-^U 



2 (Tj 2 (T, 



1 



2;rcr| a^ _x 



! ^^pj":r^ — 2[<^2^y^ + <^i^i^-yf\\'^y' 



2 a, o". 



(10.68) 



where the abbreviation z = z - /Vi - /Vj is applied. We rewrite the bracket term to 
prepare the integration, 



22, 2/- \2 ,2, 2.2 t 2~ , 2-2 

0-2 J +0-1 (Z-JJ =((7, +0-2 jj' -2(71 ZJ' + CTl Z 



/ 2 , 2,, 



\2 



+ a,'z' 



2 ^ 



(10.69) 



/ 2 , 2. 

: (cr, + (72 ) 



Therefore, /(z) reads 
1 



2 , 2 

V CTj +(72 y 



2 2-2 
(7; (72 Z 

2~~ 2~ 

(7, +(72 



/(Z)^ 



ItT (7, (72 



exp 



jexpj 



2 2 /^ 

(7, +(7, 



T . 2 , 2 . J ^'"i- T 2 2 

2 ((7, +(72 )l-» 2(7, (72 



2 , 2 

(7, +(7 



The integration can be performed by introducing the variable 



2 J 



dy. 
(10.70) 



(7, + (7, 



T 2 2 
2(7, (7, 



J- 



(7, Z 



2 , 2 

1 -2 V Cr, +(72 y 



(10.71) 



By replacing jv by s in Eq. (10.70) we obtain 
1 



f{^)- 



T 2 2 
2(7, (7, 



2;r(7,(72 y (7, +(7, 



-exp< 



2((7, +(72 )J-' 



I e ' (^.s. 



(10.72) 



The integral over exp(-,j^) is ;r"^ according to Eq. (4.70). Therefore, Eq. (10.72) 
reduces to 



/(z) = 



1 



.exp 



I v^^vj^ 1 2 2 1' 

^2n{a,~+a^^) [ 2 ((7, +(72 ) J 



(z-n,-n^f 



(10.73) 



where z = z - //[ - //2 is used. This expression shows that the PDF of the sum of 
two independent normally distributed variables is normal with mean //, + //2 and 
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variance cr, + 02- This observation can be summarized by the conclusion 

(10.74) 
5V(//, &) refers a normal PDF with mean // and variance cr^. The notation applied 
here means that X,, X2, and X^ + Xj are normally distributed with the means and 
variances specified by the corresponding 5\C 

Sum of Independent Normal Variables. The result (10.74) obtained for two 
independent normally distributed random variables X, and X^ can be extended to 
the case of any number of independent normally distributed random variables. By 
considering the two numbers considered before as one number and adding another 
number, we find that the sum of three independent normally distributed variables 
is again normally distributed. Correspondingly, we can conclude (/ = 1, AO 



Therefore, the PDF of the sum of A^ independent normally distributed random 
variables represents a normal PDF. Its mean is given by the sum of all means, and 
its variance is given by the sum of all variances. The conclusion (10.75) obtained 
can be used for deriving a corresponding conclusion for the distribution of the 
mean value of A^ independent normally distributed random variables. By replacing 
XibyXi/A^wefmdthat 



x,~5v(//„a/) ^ ±^Y.x,-^\\;Y.^i,,^t<y! 



2. 1 " 



Nt^x "N^ 



(10.76) 



Hence, the PDF of mean values is normally distributed, where the mean is given 
by the mean of all means involved and the variance is given by the mean of all 
variances involved divided by A^. The latter results were applied in Sects. 6.2 and 
6.3 for modeling a random walk (for determining the evolution of the position 
PDF in time). 



10.4 The Fokker-Planck Equation 

After considering the normal model for the joint PDF in the previous section let 
us consider now the modeling of the evolution of any PDF. This question will be 
addressed by generalizing the Fokker-Planck equation (8.21) for the PDF of one 
random variable to the case of any number of random variables. The question of 
how the Fokker-Planck equation is related to stochastic differential equations for 
the corresponding random variables will be discussed, too. 
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10.4.1 Definition of Multivariate Probability Density Functions 

The generalization of the Fokker-Planck equation (8.21) to an equation for the 
joint PDF of a vectorial stochastic process X{t) = {Xi{t), X2{t), ■■■, XJ^t)} requires a 
relevant first step: the definition of a multivariate PDF/(x, t). The most efficient 
way of doing this is the use of theta and delta functions for several variables. 

Multivariate Tlieta and Delta Functions. In Sect. 4.2.2 we introduced theta 
and delta functions of one variable. For a vectorial process X(t) = {Xi(t), X2(t), ■■■, 
Xf^t)}, the corresponding theta and delta functions are given by 

d{x-X(t)) = d{x^-X,it))d{x2 -X,(0)---^(x,v -^n(0), (10.77a) 

S{x- X(t))^ six,- X,(t))s{x2- X2{t))--S{x^ - X^(t)). (10.77b) 

Hence, multivariate theta and delta fiinctions are products of all the theta and delta 
functions of single variables. 

Multivariate PDFs. The last expression provides the basis for the definition of 
a multivariate PDF. By averaging (10.77b), the joint VDFflx, t) can be defined by 

fix,t) = {S{x-X{t))). (10.78) 

The brackets refer to the mean value defined by Eq. (4.1). The latter definition 
generalizes the definition (4.29) of the PDF of a single variable. In terms of the 
normalization property of delta functions we fmd that this definition satisfies the 
normalization condition for the joint PDF/(jc, t), 

jf(x,t)dx = \{S{x-X(t)))dx = (l) = 1. (10.79) 

Here, dx = dx, dxi-- dxj^ represents a multivariate differential given by the product 
of all differentials involved. Two-point PDFs can be defined correspondingly. For 
example, the two-point PDF/(jc, t; x', f) for having joint events (x, t) and {x', t') is 
defined by 

f(x,t;x',t') = {5{x-X{t))5{x'-X{t'))). (10.80) 

The one-point VHY f{x, i) can be recovered from this definition, 

f(x,t) = \f{x,t-x',ndx'. (10.81) 

The validity of this relation can be seen by using the definition (10.80) of the two- 
point PDF i{x, t; x\ t'), 

fix, t) = \{5{x - X{t))s{x'-X{t'))) dx'= {S{x - X{t))). (10.82) 
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A PDF fix, 1 1 x\ t) conditioned on X{f) = x' can be defined in correspondence to 
the definition (8.37) for a single-variable PDF, 

f{x\n {5[x'-X{n)) (10.83) 

= {5{x-X{t))\X{t') = x') = {5{x-X{t))\x',t'). 

In terms of this definition the one-point PDF/(x, t) can be written 

f{x,t) = jf(x,t\x\f) f{x\t')dx'. (10.84) 

This representation will be used in Sect. 10.4.3 for the derivation of solutions to 
the Fokker-Planck equation. 



10.4.2 The Fokker-Planck Equation 

Fokker-Planck Equation. Let us consider an A^-dimensional stochastic vector 
process X{t) = {X^{t), X2{t), ■■■, Xfj(t)}. This process is assumed to be Markovian 
and to have a continuous sample path. The extension of Eq. (8.21) to an equation 
for the joint PDF/(jc, t) of the process X{t) reads 

dfix,t) _ dD,{x,t)f{x,t) ^ d'D,j(x,t)f(x,t) ^^^^^^ 

Here, the sum convention is applied, this means the sum is taken over repeated 
subscripts. Equation (10.85) represents the Fokker-Planck equation for several 
variables (Fokker 1914, Planck 1917). Its coefficients Z), and Dy are given by the 
vectorial generalizations of Z)'*' and Z)'^' given by Eq. (8.22), 

Z),(jc,0= \im—(x.[t + M)-X^(t)\x,t), (10.86a) 

D,.{x,t) = ^m^-^l^X,{t + At)-X,{t)][x.{t + At)-X.{t)\\x,t). (10.86b) 

The conditional means refer to the condition X{i) = x. Equation (10.85) has the 
structure of a diffusion equation. The coefficient Z), represents a drift coefficient 
and Dy is a diffusion coefficient. The coefficient Z),y has two relevant properties, 
which are a consequence of its definition (10.86b). The first property is that Z),y is 
symmetric, this means Dy = Dj/. The second property is that Z),y is positive semi- 
definite, this means Z),y is non-negative definite. This property of Z),y can be shown 
by multiplying the definition (10.86b) with arbitrary real nonvanishing vectors c, 
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and Cj, which results in 

D.,c,c , = hm — 1\X, it + At) - X. it) 1 c, \x it + At)- X it)\c ,\x,t] 

(10.87) 

= i™olX7(^ f^'^' ^ ^0-^,(0] 4\>c,t) > 0. 

Usually, it is assumed that Z),y is positive definite, this means 

D^.CiC^ > 0. (10.88) 

The inverse matrix of Z),y will exist for this case (Pope 2000), which is relevant to 
solutions of the Fokker-Planck equation. A positive definite matrix has positive 
eigenvalues, as may be seen in the following way (Ortega 1987): Suppose that A is 
an eigenvalue of the matrix Dy and c^ is a corresponding real nonvanishing 
eigenvector, this means Dy c, = A c,. Multiplication of both sides with c, provides 
DyCjCj ^ XcjCj. Therefore, we find X = DyCjCj I (CiC,) > 0. The existence of positive 
eigenvalues is a necessary and sufficient condition for a positive definite matrix 
Dy. For positive eigenvalues we find that the three principal invariants of Dy (one 
of the invariants is the determinant det(Z)) of Dy) have to be positive: see exercise 
10.4.1. On the other hand, three positive principal invariants imply that the matrix 
Dy has to be positive definite. 

Consistency Constraint. The consistency of the Fokker-Planck equation can 
be proven by integrating Eq. (10.85) over the sample space x, 

' dt •" dx. •" dx.dx. 

The left-hand side of Eq. (10.89) vanishes: we can write the time derivative in 
front of the integral, andj{x, t) is normalized to one. The terms on the right-hand 
side can be treated by invoking the Divergence Theorem. This theorem states the 
following (Stewart 2006): Let i? be a simple solid region and let S be the boundary 
surface of E, given with positive (outward) orientation. Let L be a vector field 
whose component functions have continuous partial derivatives on an open region 
that contains E. Then 

\L-dS = \^dx. (10.90) 

S E Sx- 

The Divergence Theorem can be applied to the right-hand side of Eq. (10.89) by 
setting Z, = Z),/and Z, = d(Dyf) I dxj, respectively. By considering an infinite 
domain, the integrals on the right-hand side of Eq. (10.89) will vanish if Z, is zero 
at the surface. Therefore, the consistency of the Fokker-Planck equation (10.85) 
requires the assumption that the PDFX.'c, ^nd its derivatives vanish for |jc| — > oo. 
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Mean Equations. By multiplying the Fokker-Planck equation (10.85) with Xj, 
and integration over the sample space we obtain 

' ' dt ' ' dx, ' * dx,dXj 

To enable the rewriting of the right-hand side and to prepare the use of Eq. (10.90) 
we write this equation as 

—\x,f{x,t)dx = -\ ' ■' ' '-^^ ' ' dx+ -^D.{x,t)f{x,t)dx 
at ox- ox, 



^dx, 



dD,-{x,t)f(x,t) 

Xf. — ^ 

dx , 



(10.92) 

dx,dD,.{x,t)f(x,t) 
ax-\ ■ ax. 

dx, dx : 



The integral on the left-hand side is equal to the mean <Xi>. The validity of the 
right-hand side can be seen by distributing the derivatives by x, involved in the 
first and third terms. For Sxj. / 9x, we find 9xj- / 5x, = S^,. Here, Sj^. is the Kronecker 
symbol, which has the properties S^ =1 for / = A; and 5j^ = for / ^ k. By account- 
ing for dxf- 1 dXj = Si.j, three of the four terms on the right-hand side of Eq. (10.92) 
can be written as integrals over the surface S according to Eq. (10.90). We assume 
that the corresponding terms disappear for |jc| — > oo so that Eq. (10.92) reads 

^^^S,,lD,(x,t)f(x,t)dx = \D,(x,t)fix,t)dx. (10.93) 

The last expression is implied by the fact that S/,,- is only nonzero for k = /. The 
right-hand side represents the mean value <Dp-. Hence we find 

-A_ll = ^D,). (10.94) 

The partial derivative by t can be replaced here by the regular derivative because 
<Xi> and <Dj> are only functions of t. Hence, <Dj> determines the transport of 
means <Xj>. For that reason Z),. is called a drift coefficient. 

Variance Equations. The variance equations can be obtained by multiplying 
the Fokker-Planck equation (10.85) with x^ x„ and integrating over x, 

jx,x„ 'I^dx = -jx.x„ ^^.MfM^ , |,,,„ B%,(x,t)fM^^ 

^ " " dt •• " " dx, ^ ' " dx_8xj 

(10.95) 
This equation can be also written 

^ ' ' =h+h- (10.96) 

at 
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We wrote here the partial derivative by t in front of the integral and applied the 
definition of <Xj X>. The symbols /j and /, refer to the first and second integral 
on the right-hand side of Eq. (10.95), respectively. To calculate /i we write 

I, = - — ^^-^ — ' '■' ' dx+ — '^-^Dix,t)f{x,t)dx 

^ 5x, J 5x, (10.97) 

= \i5,.x„ + 5„,x,)D,ix,t)fix,t)dx = {X„D,) + {X,D„). 

The first rewriting identifies an integral over a derivative (the first term on the 
right-hand side), which disappears. The next rewriting accounts for d{xi, x„) / 5x, = 
Si-j x„ + (?„, X,.. The definition of means and the property of the Kronecker symbol 
4„ to be nonzero only for k = i are used for obtaining the final expression. The 
integral /j can be calculated correspondingly. 



'-j£ 



dD,{x,t)f{x,t) 

OX, 



^ ax^aA,(x,o/(.,o^ 

dX: dX : 



U^ . dD,.ix,t)fix,t) 
= - iSki^,, + S„,x,) — ■ — dx 

oxj 

^ _j SiSk.x„ . S,„x,)D„ix,t)fix,t) ^ ^ 8iS„x,. . ^,„xj ^^^,)^(^^,)^ 

dx- dxj 

= li^kAj + SJ,)D,jix,t)fix,t)dx = (Z),„) + (A„) = 2{D„,). 

(10.98) 
The last expression applies the symmetry of Z),,^,. The combination of Eq. (10.96) 
with these expressions for /[ and /j leads then to the variance equation 

^%^ = (^„A ) + (^.A,) + 2(A.), (10.99) 

where the partial derivative by t was replaced by the regular derivative. Instead of 
considering equations for second-order moments, it is more convenient to derive 
equations for the variance 

(^A-^„ ) - {{^k - {X, )) [X,. - {X„ ))) = {X,X„ ) - {X, )(X„ ). (10. 100) 

By differentiating this variance expression we obtain 

d{x,X„ ) _ d{X,X„ ) d{X, ){X„ ) _ d{X,X„ ) d{X, ) d{X„) 

dt dt dt dt dt ^ "' dt ^ '''' 

(10.101) 
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The use of Eqs. (10.94) and (10.99) implies then the following variance equations, 



d(X,X„ 



(X„D, + X,Z)„ + 2 Z)„, - Z), )(X„ - (D„ X, 



^f \ ■' "/ \ "• "/ \ «'./ \ A/\ „/ \ „/\ „/ (10.102) 



where the variance expression (10.100) is used for obtaining the last expression. 
The variance of one component is given by setting k = n. We have <Dki> > as a 
consequence of the definition (10.86b) of Z),,„. Hence, variances are produced by 
<Di,„>: a nonzero Z)^,, causes a diffusion process (the width of the PDF increases). 
For that reason Z)j,„ is called a diffusion coefficient. An equilibrium state may be 
reached asymptotically if the first two terms on the right-hand side of Eq. (10. 102) 
appear with a negative sign, i.e., if these terms model a dissipation of variance. 

Correlations. The Fokker-Planck equation (10.85) can be used to calculate the 
correlation between Xi{t) and Xf{t'). We assume that t < f ^ t + r, where r is any 
non-negative time. By following the derivation of the corresponding correlation 
(8.33) for the case of one variable (see exercise 8.3.1), the correlation ofXi{t) and 
Xj{t + r) is found to be determined by the equation (see also Eq. (10.124)) 



^ -L={x,it)Dj{Xit + r),t + r)). (10.103) 



dr 

Thus, the correlation is unaffected by the diffusion coefficient £),y, i.e., correlations 
are not produced, but they relax according to the model provided by Dj. 



10.4.3 A Solution to the Fokker-Planck Equation 

Equation Considered. Let us illustrate the application of the Fokker-Planck 
equation (10.85) and demonstrate characteristic solution properties by considering 
an example that enables the derivation of an analytical solution. The equation con- 
sidered is a vectorial generalization of Eq. (8.34) for a single variable, 

^^--^ G,(0 + G,,(OU -(JT, /(x,0 + h— . (10.104) 

at ox. ox. ox. 

The drift coefficient Z), is a linear function of the variables x, which may be seen 
as first-order Taylor series of Z),. The inclusion of <A^i> in Eq. (10.104) defines G,j. 
as the coefficient that controls the intensity of fluctuations about the mean <Xj>. 
This linear model for Z), is well suited for the characterization of near-equilibrium 
processes. The diffusion coefficient Z),y is assumed to be only a function of time. 
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which is a convenient choice with regard to many applications. Z),y is assumed to 
be positive definite. Equation (10.104) will be combined with the assumption of 
natural boundary conditions, this means J{x, ?) — > as |x| — > oo. 

Solution Approach. Solutions /(jc, t) to the Fokker-Planck equation (10.104) 
will depend on the initial PDFy(jc', f), which has to be provided. The influence of 
the initial PDF can be treated separately from the solution of the Fokker-Planck 
equation, which is very helpful for using solutions for a variety of initial PDFs. 
This can be achieved by using Eq. (10.84), which represents the one-point PDF 
J{x, t) in terms of the PDF conditioned on the initial condition X(t') = x', 

f{x,t) = jf{x,t\x',n f{x',t')dx'. (10.105) 

The idea of this approach is to calculate a general expression for the conditional 
PDF f{x, t I x\ f) independent of the initial PDF J{x\ f), and to calculate then the 
PDF/(jc, t) by integration of Eq. (10.105). But how can we calculate /(x, t \ x\ t)l 
In terms of Eq. (10.105), the Fokker-Planck equation (10.104) can be written 



= \\ -^ gl""' + ^[g, (0 + G,,(0(x, - {X,))\f{x,t\x',n 



d^DM)f(x,t\x',f)\ 

dx-Sx- 

Hence, the conditional PDF fix, t \ x\ f) has to satisfy, too, the Fokker-Planck 
equation (10.104), i.e., the conditional PDF/(jc, t \ x\ f) has to satisfy the equation 

dfix,t\x',t') _ d[G,it) + G„{t)[x,-{X,))\fix,t\x',t') 

dt dx; 

(10.107) 
^ yfi 1 1 Y t \ Y' r ) 



^d'D,.(t)f{x,t\x',f) 



dx-dx- 
Eq. (10.83) provides the initial condition for the conditional VT)¥ j{x, t \ x', f), 

, {s{x-xin)s{x'-x{n)) _ {s(x-x')s{x'-x(n)) 

{S{x'-Xit'))) {S{x'-X{t'))) (10.108) 

= S{x-x'), 

where the sifting property of delta functions is used. 

Conditional PDF Calculation. The conditional PDF is a normal PDF for the 

single-variable case (see Sect. 8.3.2). Therefore, we may assume that/(x, t \ x\ t') 
also is given by a normal PDF (an A^-dimensional normal PDF for our case). 



f(x,t\x',t') = ,. ^„/,-— - exp^ - ^/3-\j{x, - a,){x. - al (10. 109) 
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Here, «, are the mean values and Py represent the elements of the variance matrix, 
which is positive definite and symmetric (/^y = /?,,). Therefore, the inverse matrix 
y5 ',y does exist, and it is symmetric, i.e., /? ',, = /3'^j,. Another view of looking at 
the assumption (10.109) is the following one: we ask under which conditions it is 
possible to have a normal PDF as solution of a Fokker-Planck equation. To prove 
the suitability of the assumption (10.109) we have to show that Eq. (10.109) can 
satisfy Eq. (10.107). This fact is proven in terms of exercise 10.4.4. It is found that 
the model parameters a, and /^y have to satisfy the equations 

^ = G,+G„(«,-(jr,)), (10.110a) 

^ = G„p,. + G.,p,^ + 2D,. . (10. 1 10b) 

These initial conditions for «, and Py are given by 

a,{t') = x'., (10.111a) 

Py{t') = Q. (10.111b) 

Means and Variances Implied by tlie Foliker-Plancli Equation. Next, let us 

have a look at the means and variances of/(jc, t), which are implied by the Fokker- 
Planck equation (10.104). The simplest way to obtain these equations is to specify 
the general Eqs. (10.94) and (10.102), which are valid for every Fokker-Planck 
equation, 

^-^ = G„ (10.112a) 

at 



GjX,„X„ + GiX„,X, + 2D,„. (10. 1 12b) 



d{X,X„ 

I, " /ill! \ m n I ~ iiiJi \ 

at \ / \ 

Equations (10.1 12) are similar to Eqs. (10.1 10) for the parameters ak and Ph, of the 
conditional PDF. To see the difference, we apply Eqs. (10.110) and (10.112) for 
deriving the following equations 



d_ 
dt 

d_ 
dt 



(«,-(X,))=G,„(«„-(X„)), (10.113a) 

(a„ - (X,X,))= G„„ (/?,,„ - {Xj,)]+ G„,„ (/?,,, - {Xj,)]. (10.1 13b) 



The coefficient G^,,, is usually provided with a negative sign to model a relaxation 
of fluctuations. For this case, «j and /?j.„ relax to the means and variances of/(x, t): 
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the stationary values of a,, and /?j,„ for which the left-hand sides of Eqs. (10.113) 
are zero, are given by 

a,={X,), (10.114a) 

P^.={X,X\. (10.114b) 



For this case, the conditional PDF f{x, t \ x\ t) is independent of x' because its 
parameters are independent of x'. Equation (10.105) reveals that the PDF/(jc, t) is 
then equal to the conditional VDV fix, t \ x\ f). Therefore, the unconditional PDF 
fix, t), which may have any shape initially, does relax (independent of the initial 
conditions) asymptotically to a normal PDF. 



10.4.4 Stochastic Differential Equations 

Stochastic Differential Equations. In analogy to the discussion of the rela- 
tionship between the Fokker-Planck equation (8.21) and the stochastic differential 
equation (8.55) for a single variable, let us consider now the corresponding rela- 
tionship for several variables. For the case of an A^-dimensional stochastic process 
X{t) = {X^{t), X2(t), ■■■, XjJ^t)} we generalize the Markovian stochastic equation 
(8.55) by the equation 

^{t) = a,{X,t) + h„{X,t)^{t). (10.115) 

at at 

Here, the coefficients ai(X(t), t) and bji.{X{t), t) are any deterministic functions of 
X{t) and t. The normally distributed vectorial process dWi,l dt is characterized by 

^(0\ = O, (10.116a) 

^(0^(^'))-<?.,<?(^-O. (10.116b) 

Relation (10.116a) corresponds to Eq. (8.57). Relation (10.116b) corresponds to 
Eq. (8.60) for k = n. For k ^ n this relation means that dWf,l dt is uncorrelated to 
dW„l dt. The process dW,^l dt is assumed to be independent oiX{t^. Due to the fact 
that the change of the stochastic process X{t) is fully determined by aj{X{t), t), 
bi^(X(t), t), and dWJdt, we find that the equation system (10.115) describes the 
evolution of X{t) as a Markov process: the future of the statistical properties of 
X(t) is fully determined by the present state. 
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Stochastic Difference Equations. The representation of the stochastic differ- 
ential equation (10.115) as a stochastic difference equation is relevant, e.g., to the 
numerical solution of Eq. (10.115) and regarding the derivation of the relationship 
to the Fokker-Planck equation (10.85): see the discussion in the next paragraph. 
To address this question we integrate Eq. (10.1 15) from tto t + At, 

X,{t + M)-X,{t)= \a,{X{s),s)ds+ \ b,Xx{s),s)—^{s)ds, (10.117) 

, as 

where At is a sufficiently small time interval. As for the single-variable case we 
use the Ito definition of stochastic integration, i.e., we approximate aj{X{s), s) and 
bjil(X{s), s) by their values at the lower bound t Then, Eq. (10. 1 17) can be written 

X,{t + At)-XXt) = a,{X{t),t)At + h„{X{t\t)AW,{t\ (10.118) 

where AW,-{t) is defined by 

f+Ar Jur 

AW,Xt)= \ '-{s)ds = WiXt + At)-W,{t). (10.119) 

, ds 

The properties oiAWiit) can be derived in terms of Eq. (10.1 16), 

{AW,[t))= \(^{s)ys=Q. (10.120a) 

= 5,„\ \ 5{s'-s)ds'ds = 5,„\ \ ^\ ' ds'ds (10.120b) 

t t' t t' ^^ 

= S,, I [dit'+At -s)- 0{t'-s)] ds = S„ At I ^^ ^ ^ ^, 

Here, f changes by At as does t, this means t' = t + k At, where k^ 0, ±1, ±2, .... 
Thus, the integral in the last line is only nonzero and equal to At if t' = t, which 
explains the final result of Eq. (10.120b). 

Relationship to Foldier-Plancli Equation. The question about the relationship 
to the Fokker-Planck equation (10.85) can be addressed by the calculation of the 
first two coefficients of the Kramers-Moyal equation (which has to be written for 
the case of many variables). According to Eq. (10.86), these coefficients become 

D.{x,t)= lim—(^X,{t + At)-XXt)\x,t), (10.121a) 

D,.ix,t)=lim^[[xXt + At)-XXt)][X-{t + At)-X.it)]\x,t). (10.121b) 
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The use of Eq. (10.118) in these expressions leads to 

D,{x,t)= lim —(a.{X{t),t)At+b.,{X(t),t)AW,{t)\x,t) = aXx,t), (10.122a) 

A,(x,0 = \im-^{[aXX(t),t)At+ b,iX(t),t) AW,(t) ] 
At-^o 2At 

[aj{X(t),t)At+b.,{X(t),t)AW„{t)]\x,t) (10.122b) 

where the properties (10.120) of AW,^. are used. The corresponding calculation of 
higher-order coefficients of the multivariate Kramers-Moyal equation leads to the 
same conclusion as obtained for the single-variable case: all these coefficients are 
zero because they are of higher order in At. Thus, the stochastic Eq. (10. 115) does 
imply uniquely a Fokker-Planck equation that determines the PDF evolution. 
However, a Fokker-Planck equation does not fully determine a stochastic differ- 
ential equation in general. For A' variables, Eq. (10.122b) provides N{N+ l)/2 
equations for A^^ elements of b^i (e.g., for A' = 6 there are only 21 equations for 36 
elements of Z),y). Therefore, the coefficients of the stochastic Eq. (10.115) are only 
uniquely determined by the Fokker-Planck coefficients Z), and Z),y if bjj is assumed 
to be symmetric so that only N{N+ 1 ) / 2 elements of bjj have to be determined. 

Correlations. It was shown in the previous paragraph that the stochastic dif- 
ferential equation (10.115) is consistent with the Fokker-Planck equation (10.85) 
with regard to the one-point statistics (i.e., the PDF, means and variances), but the 
corresponding consistency regarding the correlation dynamics is not demonstrated 
in this way. To address this question we use the stochastic differential equation for 
deriving an equation for the correlation function of X^{t) and Xj{t + r), where r is 
any non-negative time. For doing this we consider 



d(X,(t)Xj{t + r)) I dXAt + r) 



tJr \ dt 

ar \ ai I (10.123) 

= (i, (0 «, [X(t + r), t + r)) + Ix, (0 b., {X(t + r),t + r)^ {t + r)\. 

The last equation arises from the use of Eq. (10.115). The noise term dW/J dt{t + r) 
is independent of AXO and X{t + r) because only noise at times before t and t + r 
can affect X(?) and A^^ + r), respectively. Thus, the last term is zero and we obtain 



d{X,{t)Xj{t + r)) /^ ^, ,, 

^ ' 'X^{t)di.{X(t + r),t + r)). (10.124) 



dr 
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Here, Xj and a, are replaced by the corresponding fluctuations because the means 
oiXj and a, do not affect the result. The result obtained generalizes Eq. (8.72) to 
the case of several variables, and it recovers Eq. (10.103) if Z), = a, is taken into 
account. In this way, the consistency between the stochastic differential equation 
(10.115) and Fokker-Planck equation (10.85) is also demonstrated regarding the 
implied correlation dynamics. 



10,5 Molecular and Fluid Motion 

The mathematical modeling of molecular and fluid motion is a problem that is 
relevant to a huge variety of processes in nature (e.g., atmospheric dynamics) and 
technology (e.g., reactor chemistry). Actually, we consider only one process: the 
motion of molecules of a fluid. The difference between the terms molecular and 
fluid motion is given by the scale considered. With regard to molecular motion we 
are interested in an understanding of elementary processes with a typical length 
scale of about 10 ' m, whereas the consideration of fluid dynamics means to look 
at processes with a typical length scale of about 10"^ m. Fluid dynamic variables 
represent means of molecular variables. For example, the mean molecular velocity 
is equal to the fluid dynamic velocity. Therefore, we will derive here the equations 
for fluid motion as the moment equations that are implied by a stochastic model 
for the molecular motion. From a mathematical point of view, the goal of this 
section is to illustrate the application of stochastic differential equations and the 
Fokker-Planck equation. In particular, the goals are to show: 

• the typical structure of stochastic differential equations for a real problem, 

• the problem related to the numerical solution of such stochastic equations, 

• the use of analysis tools for deriving moment equations, 

• the typical closure problem of moment equations, 

• a consistent and systematic way to develop closed moment equations, 

• ways to assess the range of validity of different moment equations. 

The focus here is on the modeling problem, this means the derivation of closed 
equations for molecular and fluid motion. Unfortunately, the equations obtained 
cannot be solved analytically, and numerical solutions turn out to be extremely 
expensive. Interested readers may find more information about solutions of these 
equations elsewhere (Pope 2000, Heinz 2003, 2004, Fox 2003, Givi 2006, Jenny 
et al. 2010). The modeling problem will be considered here in its simplest form, 
this means without accounting for additional variables (like mass fractions of 
chemical species) or forces (like the gravity force). Such modifications, which 
may be relevant to applications, can be taken into account by following the meth- 
odology to be presented in the following. 
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10.5.1 Molecular Motion Model 

Stochastic Molecular Motion Model. Attention will be restricted here to the 
case of monatomic fluids, which do not have internal degrees of freedom (rota- 
tional or vibrational energy). The molecules are assumed to move independently. 
This corresponds to the consideration of a perfect gas. The state of each molecule 
is completely described by its position x,* and velocity V'. Here, the subscript 
i=\, 3 indicates the three position and velocity components in physical space. 
The equations considered for x* and V* are given by (Heinz 2003, 2004, 2007) 

^ = F,, (10.125a) 

at 

^y' v' -U IAb dW 

^^ = -^ ^+ P^^. (10.125b) 

dt T V 3r dt 

Here, dWj I dt is the derivative of a Wiener process. This model involves three 
parameters: (7, is the mean molecular velocity, e is the specific kinetic energy 
(it has the dimension of a squared velocity), and r is the characteristic time scale 
of molecular fluctuations. All the three parameters may depend on time t and the 
position of a molecule (the model parameters are functions of the position x in 
physical space, where x is replaced by x*{t) in Eqs. (10.125)). The apphcation of 
this model does only require the definition of the time scale r, because (7, and e 
can be calculated from molecular properties (by taking the mean over velocities 
and squared velocity fluctuations). An external force is not considered here for 
simplicity. The model considered represents an extension of the Brownian motion 
model (6.59). A difference is given by the inclusion of the mean velocity (7, in the 
drift term here, which is assumed to be zero in the Brownian motion model (6.59). 
Equations (10.125) can be solved via Monte Carlo simulation, which enables the 
calculation of all relevant variables (like {/,) as means over particle properties. 
Nevertheless, this approach is computationally very expensive (Jenny et al. 2010). 
It is usually more convenient to consider equations for moments, which can be 
solved with lower computational cost. 

Moment Equations Implied by Stochastic Model. Similar to the analysis of 
the Brownian motion model (6.59) we can study the consequences of the stochas- 
tic model (10.125) for statistical particle properties, this means we can calculate 
the evolution of the mean particle position, velocity, and variances in time. How- 
ever, our main interest here is in fluid dynamics, i.e., the properties of the fluid at 
a fixed position and time. Such fluid dynamics properties are given by the fluid 
mass density p{x, t) and fluid velocity (7,(x, t). Equations for p(x, t) and (7,(x, t) 
can be derived as a consequence of the Fokker-Planck equation that is implied by 
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the stochastic molecular model (10.125). However, these derivations are relatively 
lengthy. Therefore, only the resulting equations are presented here. All the details 
of how these equations can be obtained are given in the appendix of this section 
(see Sect. 10.5.3). All the equations presented in this paragraph are exact conse- 
quences of the stochastic model (10.125). The evolution equations for p(x, t) and 
Ui(x, t) can be presented efficiently in terms of the substantial derivative DQ/Dt = 
dQI dt + U„,dQ I dx„, (see the discussion of this derivative in Sect. 10.1), where 
Q{x, t) can be any variable. The equations for p{x, t) and U{x, t) that are implied 
by the stochastic molecular model (10. 125) can be written then 

^ + p^^ = 0, (10.126a) 

Dt ^ex„, 

^^ + A^ + 15M^ = 0. (10.126b) 

Dt 3/7 dx- p 5x,„ 

There are two unknown variables in the last equation: the kinetic energy e{x, t) 
and deviatoric stress dipc, t). Both e and o?,, are related to the variance of molecular 
velocities. In particular, e represents the isotropic variance contribution, and djj is 
the anisotropic variance contribution (see the corresponding explanations in Sect. 
10.5.3). The stochastic model (10.125) implies an equation for the variance, which 
can be used to derive the following equations for e and d^j. 



^.^^^^^W^ + ^.„,,.^.^.0, (10.127a) 

Dt Ip 5x„, 9x„, 3 9x,. 



Dd,. ^ 1 dp{v,v^-v,x),5,^ll,)v,„ ^ dU,( ^ ^2^^ 



Dt p 5x,„ 5x,„ V "■' 3 

e5,„A--5„—-^\d^,+-e5,„,\ = — d, 



^ \ -~ mi ^ ' ~ mi \ r-i ii «, mk ^ mk ii ' 

dx„, ( 3 3 ' dx„[ 3 T ' 



(10.127b) 



The dy equation provides zero on both sides if we set / =7 and take the sum overy. 
Equations (10.127) contain again unknowns given by the terms that involve three 
velocity fluctuations u, (the triple correlation). An equation for these triple corre- 
lations can be also derived from the stochastic model. This equation reads 



Dv.vv, 1 dpv.vv.v^ 1 dpv,v„, 1 dpvv,„ l dpv,v„, 

+ —'^VjVf. VjV^ <^^^VjV, 

Dt p 5x,„ p 5x,„ p 5x,„ p 5x,„ 

dU. dU. dU, 3 

+ — ^m^i^t + — ^^,„^iVt + — ^^„,^i^i = — ^i^i^f 
dx,„ "" ' * 5x„, "' ' ' dx,„ "• ' ' T ' ' ' 

(10.128) 
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Discussion of Moment Equations. It is interesting that Eqs. (10.126) for the 
fluid mass density fix, t) and the fluid velocity (7,(x, t) are equal to equations that 
follow from several molecular motion equations, as, for example, the Boltzmann 
equation (Heinz 2003, 2004, Jenny et al. 2010). The same applies to the left-hand 
sides of Eqs. (10.127) and (10.128). Hence, the influence of the stochastic molec- 
ular model (10. 125) considered does only appear on the right-hand sides of the djj 
Eq. (10.127b) and triple correlation equation (10.128). Other molecular motion 
models provide right-hand sides of the dy and triple correlation equations that have 
the same structure (Jenny et al. 2010). The equation system (10.126)-(10.128) of 
coupled fluid dynamics equations is still unclosed due to the appearance of the 
term with four velocity fluctuations y, in Eq. (10.128). 



10.5.2 Fluid Dynamics Equations 

Next, let us consider how it is possible to overcome the closure problem of 
Eqs. (10.126)-(10.128) described in the previous paragraph, i.e., how closed 
equations for fluid dynamics can be derived. 

Algebraic Model for Fourth-Order Correlations. To close Eqs. (10.126)- 
(10.128) we need a model for the unknown fourth-order velocity correlations in 
the triple correlation equation (10.128). A corresponding closure model can be 
obtained by assuming that the velocity PDF can be approximated by a joint nor- 
mal PDF, which leads (in generalization of Eqs. (10.43b) for the fourth-order 
correlations of a bivariate normal distribution) to the following parametrization of 
fourth-order central velocity moments. 



vPjV.v^ = v.v. v,v„, + v.v, v.v„, + v,v^ v-v,. (10. 129) 

This approximation represents a reasonable assumption for all fluids that are not 
too far from an equilibrium state. It is relevant to see that this assumption does 
only affect the evolution of triple correlations, which are small for fluids that are 
close to an equilibrium state. The gradient of fourth-order correlations required in 
the triple correlation equation (10.128) is then given by 



1 dpv,VjV,v„, _ dViVj dv^v, dVjV, 



-Vi.V„. + — VjV„, + -^ — v,v„. 



p OX ox ox ox 

^ "' '" "' '" (10.130) 

1 dpv,v,„ 1 dpvp„, 1 dpv,v,„ 

-V,V,+ ^ V,Vi.+ VjV^. 



p dx„ p 5x,„ p 5x,„ 
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The use of this expression in Eq. (10.128) for velocity triple correlations leads to a 
closed equation for triple correlations, 







LJh^hjh^ Lhihj c>^,-^k , ^^.rk 




Dt 5x,„ 5x,„ ■ 5x„ 

5(7, 5(7,. 5(7, 


3 


5x,„ 5x„, 5x,„ 



(10.131) 



In this way we have derived a closed system of fluid dynamics equations given by 
Eqs. (10.126), (10.127), and (10.131). 

Algebraic Model for Third-Order Correlations. The cost of simulations can 
be reduced by using Eq. (10.131) for the derivation of an algebraic approximation 
for the triple correlations. We assume that the substantial derivative and the terms 
that contain triple correlations multiplied with velocity gradients can be neglected 
in comparison to the other terms. 



5x„, 



5x„ 



VV H VV 

' '" 5x„, ' '" 



(10.132) 



This model implies for the triple correlations in the energy equation 



2r 
3 



dvv.. 



de — 

V V H '— 

5x„ 5x„ 



2T— d(e5„„+v,v,„) 



2t 2 



3 13 



_d_ 
5x„ 



3 
-d. 



5x„ 



(10.133) 



where the variances u,u„, = 2 e / 3 5j„, + t/,,,, are represented by the specific kinetic 
energy e = vpi 12 and deviatoric stress t/„„ (see Sect. 10.5.3). In this case, we have 
a closed system of fluid dynamics equations given by Eqs. (10.126) and (10.127) 
combined with Eqs. (10.132) and (10.133), respectively. 

Algebraic Model for Second-Order Correlations. The fluid dynamics equa- 
tions can be further simplified by the derivation of an algebraic model for d^j. This 
model can be obtained by neglecting Ddyl Dt, the gradients of triple correlations, 
and the anisotropy contributions dy in the parenthesis terms of Eq. (10. 127b), 



^5(7, 5(7, 
— '- + — - 
8xj 5x, 



2^SU^ 
3 " dx. 



:dr 



(10.134) 



This relation can be written more efficiently by introducing the shear rate tensor 



S, = ■ 



5(7, 5(7,. 
— '- + — - 
dxj dx. 



(10.135) 
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and the related deviatoric shear rate tensor 





1 „ c- 1 


dU 


^ =S-- 


--S 5 =- 






3 «" '/ 2 


5x, 



dx- 



3 dx.. " 



(10.136) 



which is the deviation of the shear rate tensor from its isotropic part. According to 
its definition, 5/ has the property 5/ = 0. In terms of the definition of 5,/, the 
algebraic model (10. 134) for fi?,^ can be written 



The last expression introduces the diffusion coefficient 

re 



(10.137) 



(10.138) 



which is called the kinematic viscosity. 

Fluid Dynamics Equations. The use of the approximation (10.137) simplifies 
the fluid dynamics equations significantly. Equations (10.126) read now 



Dp_ 
Dt 



-ps„. 



DU, _ 2 dpvSj 2 dpe 



Dt 



P 5x,„ 



3/7 dx. 



(10.139a) 
(10.139b) 



In the first equation we applied 5,, = dUJ dXj, which follows from Eq. (10.135). 
The second equation is often written in terms of the viscosity // = /? v. The energy 
is given by Eq. (10.127a) combined with Eq. (10.133) for u,y,y„, . In v.v^v,,, we 
have to neglect anisotropy contributions djj in the parenthesis terms to be consist- 
ent with the approximations used in the dy equation - we have 



20z"e 5e _ 20 5e _ y de 

27 dx.„ ~ 9 " dx... ~ ^ Pr dx... 



(10.140) 



The last writing presents this expression in its standard formulation. Here, Pr is 
the Prandtl number, and y= 1 + 2 //is the ratio of specific heats, where /counts 
the degrees of freedom. Monatomic gases have/= 3 degrees of freedom such that 
y= 5/3. Therefore, we have Pr = 3/2 for the case considered (Jenny et al. 2010). 
By using Eq. (10.140) for the triple correlation and Eq. (10.137) for dy we find the 
energy equation to be given by 



Z)e 
~Dt 



LA 

P Sx,„ V 



pv 



y de 
Pr dx... 



dx... 



2 dU, 
—e 

3 dx, 



(10.141) 
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The last two terms of this equation can be rewritten in terms of the definitions of 

Sij and S/. We use again 5,, = dU,l dxj. The term involving S„,f can be written 



dU. , ^l 
dx... '"■ 2 



^dU, ^ dU,„ 2 dU, ^^ 
^5x,„ dx. 3 dx, 



Sj=SjsJ, (10.142) 



which is a consequence of symmetry properties and the fact that S/ = 0. Thus, the 
energy equation can be written as 



De _l d 
Dt~ p dx,„ 



Y de 

pv- 

Pr5x„,, 



+ 2vSj Sj-^eS,, (10.143) 



Equations (10.139) combined with this energy equation represent a closed equa- 
tion system. The equations can be presented in different ways by using the rela- 
tions between the kinetic energy e with the pressure p and temperature T, 

e = -^ = -RT. (10.144) 

2 p 2 

Here, R refers to the gas constant. 

Navier-Stokes Equations. Equations (10.139) combined with Eq. (10.143) 
represent the Navier-Stokes equations, where the ratio y I Pr is chosen according to 
the fluid considered. The value y = 5/3 is the correct value for monatomic gases, 
but a Prandtl number value Pr = 3/2 derived here as a consequence of the simple 
molecular model (10.125) needs adjustments (for most gases measurements show 
a more or less constant Prandtl number value Pr = 2/3). Which influences cause 
changes of the fluid dynamic variables? The mass density p is changed by the 
dilatation 5„, which measures compressibility. Changes of the fluid velocity (7, are 
caused by two effects: molecular diffusion (the first term on the right-hand side) 
and kinetic energy (or pressure) gradients (the last term): a decreasing pressure in 
the X, direction (i.e., a negative pressure gradient) implies a positive acceleration 
DUj/ Dt of the fluid in this direction. Changes of the kinetic energy can be caused 
by three effects. The first effect is given by molecular diffusion (the first term on 
the right-hand side). The second effect is given by viscous heating (the second 
term). This contribution is always positive. It arises from the conversion of kinetic 
energy into heat. The third effect is due to compressibility (the last term). Analyti- 
cal solutions of the Navier-Stokes equations can be only found under very specific 
conditions. Numerical solutions of the Navier-Stokes equations turn out to be 
extremely expensive if the fluid considered is turbulent, which is the usual case 
(Pope 2000). Therefore, studies of fluid properties on the basis of these equations 
usually represent a very complicated matter. A simple illustration of characteristic 
properties of the Navier-Stokes equations was given in Chap. 9 by the discussion 
of the Lorenz equations and their chaotic solutions. 
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10.5.3 Appendix: Implications of the Stochastic Molecular Model 

This section shows how the equations of fluid dynamics (10.126), (10.127), and 
(10.128) can be derived from the stochastic molecular motion model (10.125). 

Joint PDF and Conditional PDF. The joint PDF for molecular positions x, (0 
and velocities V'{i) involved in the molecular model (10.125) is defined by 

f{w,x,t)=i5(x\t)-x)5{y\t)-w)), (10.145) 

where x and w refer to the sample space positions and velocities, respectively. The 
brackets denote an ensemble average. To define fluid dynamic variables at fixed 
positions x we need the conditional PDF 

F{w,x,t)=j-r-^ ^{5(x{t)-x)5{^'{t)-w)). (10.146) 

{5\x{t)-x)y 



The delta function d{x''{t) - x) involved here is proportional to the instantaneous 
molecular mass density, which is defined by 

p'{x,t) = M5[x''(t)-x). (10.147) 

The integration of Eq. (10.147) overx shows thatM=|yC>'(A:, t)dx. Hence, Mis the 
total mass of molecules within the domain considered. The mean molecular mass 
density is given by averaging Eq. (10. 147), 

p{x,t) = {p{x,t)). (10.148) 

By applying Eqs. (10.147) and (10.148), the conditional PDF can be written 

F{w,x,t) = —}—(^p'{x,t) S(v'(t) - w)). (10. 149) 

Hence, the joint PDF/(m', x, t) and the conditional PDF F{w, x, t) are related by 
f(w, X, t) = p{x, t)F{w, X, t) I M. Expression (10.149) shows that F{w, x, t) inte- 
grates to one, this means | F{w, x,t)dw^\. 

Fluid Dynamic Variables. Integrations over F{w, x, t) provide fluid dynamic 
variables at a fixed position x and time t. For any function Q of velocities we find 

\Q(w)F(w,x,t)dw = ^-\Q{w)lp{x,t)5iy'(t)-w))dw 
p{x,t) \ ' 

= —^\{p'{x,t)Q(v'(t))s(v'(t)-w))dw (10.150) 



p{x,t) 



p{x,t)Q(v'(t))) = Q{Vix,t). 
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The first line makes use of the definition of 7^(h', x, t). In the second line, Q{w) is 
written inside the brackets and replaced by QiV{t)) according to the sifting prop- 
erty of delta functions. The normalization property of delta functions is used in the 
third line. The last expression introduces an abbreviation for the previous expres- 
sion that refers to the physical meaning of this expression: we calculate the mass- 
density weighted mean over velocities at a fixed position x and time t in this way. 
Examples for the use of Eq. (10.150) are given by the following definitions of the 
first three velocity moments 



VXx,t) = jw,F(w,x,t)dw = U,{x,t), 



ViV. {x, t) = \ w-w. F(w, X, t) dw , 



ViVjVi^(x,t) = \wiWjW,, F{w,x,t)dw. 



(10.151a) 
(10.151b) 
(10.151c) 



Relation (10.151a) relates the integral to the mean molecular velocity (7„ which is 
used in the stochastic molecular model formulation. 

Fokker-Planck Equation. Equations for the fluid dynamic variables (10.151) 
can be found as a consequence of the stochastic molecular model (10.125). The 
equation for the joint VDV f{w, x, t), which is implied by the stochastic molecular 
model, is given by 



(10.152) 
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3r 5w„5>v„ 



By multiplying this equation by M and using the relation between the joint PDF 
and conditional VY)Y,f{w, x, t) = p{x, t) F{w, x, t) I M, we find 



dpF^dpw„,F 



dt 



dx„. 



dw,„ 



■^pF.^'-^ 

2t dw,„ 



(10.153) 



Mass Density Equation. The integration of the latter equation over the velocity 
sample space w implies an equation for the mean mass density. 



dt dx„. 



■\ 



dw„, 



■-^pF.^^ 
3t dw,„ 



dw = 0. 



(10.154) 



We used here F, {x, t) = Uj{x, t) and the normalization property of 7*'. The terms on 
the right-hand side do not contribute because we have integrals over derivatives, 
which disappear at infinity. By distributing the spatial derivative we find 



Dp 8U„ 
—^ + p — - 
Dt dx 



0, 



(10.155) 
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which corresponds to Eq. (10.126a). We applied the definition of the substantial 
derivative DQ I Dt = dQ I dt + U^dQI dx„, (see Sect. 10.1), where Q{x, t) can be 
any variable. 

Velocity Equation. An equation for the mean velocity can be derived by multi- 
plication of Eq. (10.153) with w, and integration over the velocity sample space, 



dpu, ^ dpvy,,^ 

dt 



dx,„ 



= -! 



5w„ 



^,„-U,n jr I 2edpF 



w, - (7,. ^ 2e dpF 

_! LpJ7 I 



3z" 5w, 
dw = 0, 



dw 



(10.156) 



3r dw- 

where integration by parts is applied. This equation can be rewritten by splitting 
V-V^j^ into contributions due to the mean velocity (7, and deviations u, = V^ - (7, 
from the mean velocity. 



vy=u,u+v:v.„ 



(10.157) 



The last term represents the variance of the velocity distribution. The consistency 
of this relation may be seen by distributing the variance according to Vj = Uj + u,. 
The combination of Eq. (10.156) with Eq. (10.157) leads to the following mean 
velocity equation 



dpU, ^ dpUfJ^ ^ dpv,v„, _Q 
dt 5x„, 5x„, 

For any function Q{x, t) we have the relation 



df^ , dpQU„, 



dt 



dx,„ 



Q 



dp , dpU„, 
dt 5x„ 



■P 



56 , ^ dQ 



dt 



5x„ 



P 



DQ 
Dt ■ 



(10.158) 



(10.159) 



The first bracket term does not contribute here because of Eq. (10.154). By setting 
Q = Uj and using the last relation, we can write the velocity equation as 



DU, ^ 1 dpv,v„ 



Dt p 5x„ 



0. 



(10.160) 



To prepare the use of approximations (see Sect. 10.5.2) it is helpful to split the 
variance into two contributions, v,v„, = 2 e/3 c^„, + fi?„„. Here, the kinetic energy e 
and deviatoric stress dip which has the property c/,, = 0, are given by 



e = \viVi, 



d.j = vp. 



■-e5„. 



(10.161) 
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The resulting equation for (7, is then equal to Eq. (10.126b), 

:0. 



DU,^J^dp^^\ dpd,,.^ 



Dt 3p dx- p 5x„. 



(10.162) 



Variance Equation. Equation (10.160) is unclosed due to the appearance of 
the variance VjV„, . To derive an equation for i;,i;,„ we multiply Eq. (10.153) by 
(w, - [/,) {Wj - Uj) and integrate over the velocity sample space. Let us separately 
calculate the right-hand side (RHS) and left-hand side (LHS) of this equation, 



RHS = j(>v,-{/,)(w^,-C/,) 



dw,„ 



w,.. - U,„ ^ 2e dpF 
pF + — 

3r 5w,., 



m 111 



dw 



T 3r 5w,„ 

2e 



dw 



-^\{w,- U, )(w, -U^)pFdw + ^\ iS,„,S^„, + S^„A,„ ) pF dw 

2 4e 

--pv.v +—pS 

T it 



(10.163) 



where integration by parts is applied. The term on the left-hand side is given by 

'dpF , dpU„,F , dp{w„,-UJF' 



UiS = \{w,-U,){w.-U.) 



dt 



Sx„, 



5x„ 



dw 



dpv,Vj dpi] J) Pi SpvpjV,,, 5(7, 



5(7,. 



dt 



dx,„ 



dx„. 



P 



Dvpj dpvpjV,,, dU, 



P^v„Vi+p—^v„p, (10.164) 
5(7, 



Dt 



dx„. 



P — - V„XI i + P — - V„,Vi ■ 

I r^ III / I r^ 111 I 

ox,„ dx,„ 



This rewriting is obtained by using the derivatives first such that they apply to all 
the integral and adding then corrections (given by the terms that involve velocity 
gradients). The last expression results from the use of Eq. (10.159). The combina- 
tion of Eqs. (10.163) and (10.164) leads then to the variance equation 



Dv,Vj 1 SpvpjV,,, ^ 5(7, 
Dt p 5x,„ 



5(7, 

8x„ dx,„ 



2( 2e ^ 



(10.165) 



To prepare the use of approximations we split the variance into an isotropic and 
deviatoric part, V/Vj = 2e/3 4, + t/,,. For e, Eq. (10.165) implies the equation 



De 1 dpv,v,v„, dUJ, 2 ^ 
Dt 2p 5x„, 5x„ V 3 



:0. 



(10.166) 
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This equation corresponds to the energy equation (10.127a). Equation (10.127b) 
for djj can be obtained by differentiating the djj definition (10.161) and replacing 
the total derivatives of the variance and e according to Eqs. (10.165) and (10.166), 



Dt 



1 SpViVjV^ dU. 



P 5x,„ 



5x,„ 



2;. 



1 dpv.v^v,,^ , dU^ 



5x„, 



Ip 5x,„ 



5x„ I 3 



'-e5„. 



1 dp{vp.-v,v^5,^IZ)v,„ a(7, 



P 



5x„ 



'A d... 



2 



dU 



'-eS,„ 



hvx^ 



'^ d^,+^eS„„ 



(10.167) 

Triple Correlation Equation. The last two equations are unclosed due to the 

term that involves three velocity components. To derive an equation for this triple 

correlation we multiply Eq. (10.153) by (w, - (7,) (w, - Uj) (w^ - (7,.) and integrate 

this equation over the sample space. The right-hand side of this equation reads 



RHS = j(w,-C/,)(>v,-C/,)(w,-C/J 



5w„, 



w.„-U,„ 2e dpF 

3r 5w„ 



m in 



dw 



+ S,„(^.-U,)(wj-Uj)) 



in 111 



w,„ - (7,„ ^ 2e dpF 
pF -\ — 

3r dw,„ 



dw 



= — j (m', - Ui ){w. - Uj ){w, -U,)pFdw = — p u,!; u, , 

(10.168) 
where integration by parts is used. The corresponding left-hand side of the equa- 
tion considered is given by 



hm = \{w,-u,){w.-u j){w,-u,) 



dpF ^ dpU„,F ^ dp{w^-UJF 



dt 



dx„, 



5x„ 



dw 



dpvpp, dpU,„v,v^v, dpvpjV.v^ DU, DU ^ 

+ H + p -V:V,- + p -ViV^ 

dt dx.„ dx„. '^ Dt ' * "^ Dt ' * 



■P 



DU, 



dU, 



dU, 



dU, 



VV- + p — -v„v.v,. + p — -v,„vv,. + p — -v„v.v 
Dt ' ' dx„ '" ' ' 8x^ '" ' '' 5x„ '" ' ■' 



(10.169) 
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The first three terms appear as a consequence of applying the derivatives to all the 
integral, and the remaining terms are corrections. By using Eq. (10.159) for the 
first two terms and Eq. (10.160) for the substantial derivatives of velocities we 
find the expression 



LHS = p^Wi ^ 5pw^ _ 5P^— _ 5P^— 
Dt dx,„ dx,„ dx,„ 



-—Z ViVj+ p^V„,VjV,+ p-—v,„ViV, +p- V„,V,Vj. 

The combination of this expression with the RHS (10.168) then implies 



(10.170) 



Dvp^v, 1 dpvpjV.v,,, 1 dpv,v,„ 1 dpVjV,,, 1 dpv.v 



* "" V-Vj 



Dt p 5x,„ p dx,„ ' p 5x,„ ' p dx^ 
dU. 5(/. dU, 3 

C*m ''-^•nj '^■'•m ^ 

(10.171) 

which agrees with the triple correlation equation (10.128). 



10,6 Summary 

The methodological basis for the modeling of distributions of random variables 
and the evolution of PDFs and stochastic processes was presented for one random 
variable in Chaps. 4, 6, and 8. In this chapter, we extended these concepts to the 
case of joint random variables. Let us summarize the features observed regarding 
the extension of data analysis concepts, PDF modeling concepts, and concepts for 
describing the evolution of PDFs and stochastic processes. 

Extension of Data Analysis Concepts. The characterization of the properties 
of several random variables differs from the analysis of single variable properties 
by the need to account for correlations (uncorrelated variables can be treated like 
single variables). An efficient way to account for such correlations is the use of 
conditional PDFs, which are rescaled joint PDFs, and related conditional means. 
The advantage of these concepts was demonstrated regarding the optimization of 
models considered in Chap. 2: a conditional mean was shown to represent an 
optimal model, yj^x) = <Y\ x>. In addition to the approach presented in Chap. 2, 
this relation enables the development of optimal models by the calculation of the 
conditional mean on the basis of data, this means without the use of any modeling 
concepts. Other illustrations of the benefits of conditional moments can be found, 
e.g., in Klimenko & Bilger (1999) with regard to turbulent combustion problems. 
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Extension of PDF Modeling Concepts. Which modeling concepts can be used 
to describe the joint statistics of correlated variables? This is a nontrivial question, 
because many PDF types for single variables cannot be extended straightforward- 
ly to the case of several variables. Here, the most relevant case was considered: it 
was shown that the normal PDF model for a single variable can be extended to a 
joint normal PDF model for several random variables that accounts correctly for 
any correlations. The applicability of this concept to any case considered can be 
proven in two ways: by showing that the normal PDF moment relations (10.43) 
are satisfied, or by demonstrating that scatter plots of the joint PDF agree with the 
consequence of a joint normal PDF (given by the elliptical shape of isolines in the 
(x', j')-coordinate system). The first way is helpful for showing that models (like 
the Brownian motion model considered in Sect. 6.4) have a joint normal PDF. 
The second way is usually applied for analyzing the joint PDF of real data (like 
the atmospheric velocity and temperature statistics discussed in Sect. 4.5). The 
joint normal PDF model does often provide the basis for modeling concepts. This 
was illustrated here by means of two examples: First, it was shown that the joint 
normal PDF model justifies the use of linear optimal models. Second, the formu- 
lation of a random walk as a sum of jointly normally distributed contributions was 
shown to represent a sound model: it implies a random walk process that evolves 
normally distributed in time, which is the typical feature of a diffusion process. 

Extension of PDF Evolution Concepts. How is it possible to extend concepts 
for the evolution of PDFs of single variables to the case of several variables? It 
was shown that the Fokker-Planck equation for the PDF evolution and stochastic 
differential equation discussed in Chap. 8 can be extended to the case of several 
variables. As given for the single-variable case there exists a unique relationship 
between the Fokker-Planck equation and stochastic differential equation provided 
the coefficient of the noise term in the stochastic equation is a symmetric matrix. 
This relationship is helpful for the numerical Monte Carlo solution of diffusion- 
type partial differential equations that cannot be properly solved on the basis of 
other solution techniques. Consistent with the corresponding finding for single 
variables, it was shown that the Fokker-Planck equation for several variables can 
be solved analytically if linear dynamics of random variables are considered. The 
application of the PDF evolution equation presented here to the modeling of fluid 
dynamics in Sect. 10.5 illustrated the typical structure of stochastic models for a 
real problem and the typical problems related to the calculation of the solution of 
such equations: the numerical solution of the PDF evolution equation via Monte 
Carlo simulation is computationally expensive, and moment evolution equations 
are unclosed due to the appearance of higher-order correlations. A consistent and 
systematic solution for such closure problems was demonstrated by the derivation 
of closure models that are based on the PDF evolution equation. 
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10,7 Exercises 

10.2.1 Use the definition /(x, y) = <S{x-X) S[y-Y)> of a joint PDF to show that 
every joint PDF J{x, y) of unbounded variables x and v has the following 
properties. Here, g(x,y) can be any function of x and j. 

a)fix,y)>0, 

b) f{-'x>,y) = /(oo, j) = fix -co) = /(x,oo) = 0, 

c)\\f(x,y)dxdy^l, 

d) \\ g{x, y) fix, y) dx dy = {g{X, Y)) . 

10.2.2 Consider the definition of the conditional mean 

{g{X,Y) I x) = -±^{g{X,Y)5{x-X)). 
/(x) 

Specify this definition for the case that X and Fare independent variables. 

10.2.3 Consider the optimal model yjJix) = <Y \ x>, which was derived in Sect. 
10.2.3. Assume that the joint PDFy(x, y) of any data set is available as the 
result of measurements. 

a) Explain how the optimal model yjx) can be calculated on this basis. 

b) Explain the difference between this approach for developing an optimal 
model and the approach applied in Chap. 2 to find an optimal model. 

10.2.4 A stochastic model for Y, which provides the correct mean <Y> and condi- 
tional mean <Y\ x>, is given by 



(y) + r„.(y' 



uiX- X) 






a) Calculate <Y^> and <XY>on the basis of this stochastic model for Y. 

b) Use the results for <7^> and < J^7 > to explain under which condition 
the model for Y can represent a reasonable model. 

10.3.1 Consider the model (10.39) for the joint FB¥f[x, y). 

a) Integrate y(^, y) to show that |/(x, y) dy ^fix). 

b) Use this result to explain why the condition Ifx, y) dx =f{y) is satisfied. 

10.3.2 The conditional PDF/(y | x) is given by Eq. (10.47). Considered as a func- 
tion of y ,f(y I x) is a normal PDF with mean rxy x and variance 1- rxy^, 
which is divided by <y^>"^. Use this fact and the known properties of a 
normal PDF to show that Ify \x)dy^ 1 . 
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10.3.3 Consider the conditional PDF/(y |x) given by Eq. (10.53). 

a) According to Eq. (10.19), the global variance is defined by multiplying 
the conditional variance <[Y- <Y\ x>]^ \ x> with the PDFy(x) and inte- 
grating over the sample space x. Show that this global variance is equal 
to the global variance <[Y-<Y\x>^xT'^ considered in Sect. 10.2.3 (see 
the error (10.38)). 

b) Calculate the conditional variance <[Y - <Y\ x>]^ \ x> and the global 
variance <[Y-<Y\x>-,,x]^'' ^s fonctions of ry,'. 

c) Explain why the conditional variance <[Y- <Y\ x>]' \ x> is found to be 
equal to the global variance <[Y-<Y\x>^^xf^- 

10.3.4 Consider the ellipse equation x'^ / a^ + y'^ / b^ = 1. Here, a and b are given by 
Eq. (10.60). Specify the ellipse equation for rj^y— >• 1 and r^— > -1. 

10.3.5 The table shows the correlation coefficients of velocity components (m and 
V are horizontal velocities and w is the vertical velocity) and the tempera- 
ture T. The data were obtained by measurements for different stabilities in 
the atmospheric surface layer (see the discussion in Sect. 4.5). 



Stable Case: 


-0.66 


-0.04 


-0.11 


0.84 


-0.71 


0.02 


Neutral Case: 


-0.01 


-0.26 


0.18 


0.39 


0.18 


-0.18 


Unstable Case: 


0.16 


-0.13 


-0.02 


0.01 


-0.11 


0.50 



a) Identify one case that is basically characterized by horizontal motions. 
Explain your reasoning. 

b) Identify one case that indicates significant upward motions of warm air. 
Explain your reasoning. 

c) Explain for each of the three cases considered which variables have to be 
accounted for in a stochastic model that characterizes the most basic 
features of the flow considered. 

10.3.6 According to Eq. (10.65), the PDF J[z) of the sum Z = X+ 7 of any two 
random variables X and 7 is given by/(z) = \/(x, z-x)dx. 
a) The joint PDFy(x, y) is assumed to be the normal PDF of two correlated 
variables X and Y. Show that/(z) is given for this case by 
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b) Replace in the/(z) formula the statistics of X and 7 by the statistics of Z. 
Explain the meaning of this rewriting. 

c) A nonzero correlation <XY> rnay lead to a lower variance of Z than 
given for the case of uncorrelated variables X and Y (for which we have 
<XY > ^0). How is it possible to understand this observation? 

10.3.7 Consider two independent random variables X and Y, which are uniformly 
distributed on the interval [0, 1]. 

a) Calculate the PDF of the sum Z = X+Y. 

b) Compare the result obtained in a) with the conclusions of Sect. 4.4.3 
(see Fig. 4.14). What will be the PDF of a sum of a large number of in- 
dependent variables that are uniformly distributed on the interval [0, 1]? 

10.4.1 The principal invariants of the symmetric matrix Z),y are the following once: 

III = ]rDAA, - \dA.A,, + \d.„D,A = det(Z)). 
6 2 3 

a) Calculate the three principal invariants in principal axes as functions of 
the eigenvalues A^ A2, and Aj. Show that /, //, and /// are positive if the 
eigenvalues Ay A^, and A^ are positive. 

b) The three principal invariants and eigenvalues are related via the cubic 
characteristic equation A^ - 1 A} + II A- HI = 0. Use this equation to show 
that the eigenvalues are positive if/, //, and /// are positive. 

10.4.2 Show for any matrix Pjj{t) the validity of the relation 

dp-\j „-, dp,„ , 



dt dt 



which will be apphed in exercise 10.4.4. Hint: differentiate /3^„ /T' = 5^, 



10.4.3 Show for any symmetric matrix /^y(?) the validity of the relation 
1 fl'det(/?) _ __, dp,_ 



p- 



det(y5) dt dt 

which will be used in exercise 10.4.4. The validity of the latter relation can 
be shown by considering df{x, t)l dt of the joint normal PDF 



/(■^'O = 7r^XF77^7^==^exp|--;5-',y(x, -a,)(A-, -a,)l 



j_ 
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and integrating df{x, t) I dt over the sample space x. Here, «,(?) are the mean 
values and Piff} represent the elements of the symmetric variance matrix. 
Hint: you have to use the relation shown in exercise 10.4.2. 

10.4.4 Consider the Fokker-Planck equation (10.107) for the conditional PDF 
f{x, 1 1 x', t') combined with the initial condition /(jc, f \ x', f) = 5{x-x'). 

a) Calculate the partial derivatives of the conditional PDF (10.109), which 
appear in Eq. (10.107). Hint: use the relations shown in exercises 10.4.2 
and 10.4.3 to simplify the analyses in b) and c). 

b) Show the conditions under which the conditional PDF (10.109) satisfies 
the Fokker-Planck equation (10.107). 

c) Show the conditions under which the conditional PDF (10.109) satisfies 
the initial con6i\\.\onj{x, t' \ x', t') = 5{x-x'). 

10.4.5 Consider the relationship between the Fokker-Planck equation (10.85) and 
stochastic differential equation (10.1 15) discussed in Sect. 10.4.4. 

a) Use this relationship to determine the evolution equation for means that 
is implied by the stochastic differential equation (10.1 15). 

b) Use this relationship to find the evolution equation for variances that is 
implied by the stochastic equation (10. 115). Write the model parameters 
in this equation in dependence on the stochastic process X{t) and t. 

10.4.6 Consider the relationship between the Fokker-Planck equation (10.85) and 
stochastic differential equation (10.115) discussed in Sect. 10.4.4. 

a) Explain for which purpose it is particularly helpful to use the stochastic 
differential equation (10. 115). 

b) Explain for which purpose it is particularly helpful to apply the Fokker- 
Planck equation (10.85). 

10.5.1 Consider the stochastic velocity model (10.125). We assume that (7„ e, and 
rare constants. 

a) Find the equation for the velocity variance < v' (?) F^* (?) >■ 

b) Solve this variance equation. 

c) Explain the characteristic features of velocity variances as ? — >• co. 

10.5.2 Consider the stochastic velocity model (10.125). We assume that (7„ e, and 
rare constants. 

a) Find the equation for velocity correlations < V*{t) V^{t + r) >, where r is 
any non-negative time. 

b) Solve the equation for velocity correlations < V'(t) V,^\t + r) >. 

c) Calculate the velocity correlations of unequal velocity components (this 
means for / ?^ ^) for ? — > oo by taking reference to the results obtained in 
exercise 10.5.1. 



10.7 Exercises 439 



10.5.3 The stochastic model (10.125) for molecular velocities provides uncoupled 
equations for the components of velocity. On the other hand, the variance 
equations (10.165), which are implied by the stochastic molecular velocity 
model (10.125), predict couplings between all velocity components (i.e., 
nonzero cross variances). Let us assume that the velocity variances were 
isotropic at any initial time: explain why the variances may become aniso- 
tropic after some time (what is the reason for the development of couplings 
between different velocity components?). 

10.5.4 The table shows fourth-order moments of wind velocity components {u and 
;; are horizontal velocities, and w is the vertical velocity) measured in the 
atmospheric surface layer for a neutral stratification. 50,400 sample values 
are available: see the description of these measurements in Sect. 4.5. The 
corresponding value found by using the normal parametrization (10.129) of 
fourth-order moments is given in the <>^, column. The ratio of fourth-order 
moments to the corresponding normal parametrization value (10.129) is 
shown in the <> /<>v column. 







On 


<>/<>A. 






On 


<>/<>A. 


<M^> 


0.9241 


0.8902 


1.0384 


<uvw~>^ 


-0.0045 


-0.0053 


0.8497 


<U^ V> = 


-0.0299 


-0.0093 


3.2212 


<uw^> = 


-0.0323 


-0.0260 


1.2408 


<U^ w> = 


-0.1175 


-0.1116 


1.0529 


<v'^> 


0.2612 


0.2295 


1.1381 


<u^v^> = 


0.1625 


0.1507 


1.0782 


<v^w> = 


0.0270 


0.0281 


0.9624 


<u^ vw>^ 


0.0177 


0.0192 


0.9196 


<v^w^> = 


0.0428 


0.0374 


1.1433 


<u^w^> = 


0.0886 


0.0785 


1.1286 


<v\v'> = 


0.0129 


0.0129 


0.9981 


<uv^> = 


-0.0051 


-0.0047 


1.0917 


<w'^> 


0.0608 


0.0484 


1.2562 


<M V- W> = 


-0.0200 


-0.0193 


1.0364 











a) Comment on the accuracy of data by taking reference to the number of 
available samples. 

b) Calculate the mean value of all olOfj values by neglecting the <iv' v> 
value. 

c) Do these data provide support for the normal parametrization (10.129), 
which was used for the derivation of fluid dynamics equations? 

10.5.5 Consider the stochastic velocity model (10.125). Assume that the positions 
x' and velocities V' are combined to a six-dimensional vector Z= (x*, V). 
In matrix notation, Eqs. (10. 125) can be written then 

^ = «+C7(Z-(Z)) + 6^. 
dt ^ ' dt 

Here, a is a six-dimensional vector, and G and b are 6x6 matrices, 
a) Specify a, G, and b according to the equation system (10.125). 
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b) What are the requirements for the coefficients a, G, and b under which 
the solution approach for Fokker-Planck equations, which was described 
in Sect. 10.4.3, can be used for the calculation of the velocity-position 
joint PDF/(m', X, t) related to Eq. (10.125)? 

10.5.6 Consider the stochastic molecular velocity model (10.125). The asymptotic 
change dV' I dt can be considered to be small compared to the right-hand 
side of Eq. (10.125b). The asymptotic velocities V' = dx' I dt are described 
for this case by the equation 



dx, ^^ 4 dW, 

— - = U. + J-er — '-. 

dt V3 dt 

The position PDF/(jc, t), which is related to x*{i), and the conditional PDF 
j{x, 1 1 x\ f) are related hyfix, t) = lf{x, t \ x\ f) f(x\ f) dx'. 

a) Determine y(x, t \ x', t') by applying the solution approach for a Fokker- 
Planck equation described in Sect. 10.4.3. It is assumed that U, e, and t 
are constants. Simplify /(x, t \ x', f) as much as possible by using the ex- 
pressions obtained for the parameters oiJ{x, t \ x\ f). 

b) Determine the asymptotic conditional PDF/(jc, t \ x', t') as ? — > oo. 

c) Calculate the corresponding asymptotic position PDF/(x, t). 
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